62 comments

[ 4.9 ms ] story [ 125 ms ] thread
If fake accounts can be detected, how come they can still win the presidential nomination?
Because they paid real competition to not compete and their opposition is a socialist?
Is this a matter of, "And now they'll change those patterns", or is this something that can be generalized regardless of how the fakes change their posting styles? The last lines of the article suggests to me that the answer is something like: This isn't going to work out in the world, only in the lab.
The article mentions that services already exist that are stealthy enough to fly under the radar:

"The issue with these methods, however, is that stealthier (and more expensive) like farms — which likely do not rely on fake/compromised accounts — can successfully circumvent them, as a result of spreading likes over longer timespans and liking popular pages to mimic normal users. Our recent preliminary study confirms this hypothesis on accounts used by BoostLikes.com, showing that tools similar to those deployed by Facebook (which rely on graph co-clustering) fail to accurately detect fraud."

So in essence, this is just going to slightly alter the landscape in terms of which of these services gets the money and the clients.
I think it just drives the cost per like up. I suppose there may be some threshold where the price is higher than the benefit, so perhaps that's the goal?
Good point, and it probably doesn't have to get too high for it to become prohibitive.
Would the title of this get flagged?
We reverted the title to the article's because it was neither misleading nor linkbait, and so by the HN guidelines shouldn't have been changed.

(The submitted title was "Fake Facebook accounts can be detected by when they post and the vocabulary used".)

https://news.ycombinator.com/newsguidelines.html

I get one legitimate Facebook friend request for 20 fake ones. Would be nice if Facebook was better at detecting and pruning these people.

Same goes for their other product, Instagram. I want people to find my pictures I post on there but the amount of notifications I get not from interactions but from spam accounts adding me and liking my photos is very annoying.

It's a little sad that I use the Instagram report-spam feature often enough that I become aware of subtle changes to its UI over the course of months.

I feel bad for whoever has to handle the reports on the other side.

That's interesting, because I get maybe 1 fake Facebook friend request per year?

I think a lot of the problems people have with Facebook are based in how Facebook gets used. If you have 400 friends, you're simply not getting the same experience out of Facebook as a person with 60 friends is getting.

>That's interesting, because I get maybe 1 fake Facebook friend request per year?

Maybe he does too

> That's interesting, because I get maybe 1 fake Facebook friend request per year?

I get a few more than that, but the pattern is always the same. Profile picture is of an arguably attractive girl and we have a mutual friend. The mutual friend, in my case, is always one of a few guys who historically hasn't done well with women.

I used to get a "fake" friend request a week on average with FB. Someone told me to hide my friends list entirely, meaning that I even my own friends can't see who I am friends with. As soon as I did that, I think I have gotten one or two fake ones in the last year. My only guess is that my friends' accounts are getting hacked and the hackers are just networking through their friends list and friend of friends lists and adding people along the way.

As for IG, the amount of spam accounts is almost unbearable. To me it is next to Twitter with the number of accounts. IG I average about 3-4 spam (porn or sell me followers) accounts per day. With Twitter it seems to come in chunks. I'll go a few weeks without any and then one day I will get 50-60 follower notices from obvious spam accounts. With both it makes the user experience unpleasant.

If they're able to tell before friending you that your friends list was blocked your account would be less valuable since they couldn't continue on to your friends.
I looked at 10,000 toy bricks and they were all green so I can say with great confidence that "All toy bricks are green."

That's an example inductive reasoning. It's quite flawed as a way of extrapolating from known information in to the unknown because it doesn't account for the things you don't know. In the same way, saying "Fake Facebook accounts post with this frequency or these few words" doesn't work, because it only considers the accounts that do. It can't detect the bots that don't.

Inductive reasoning is how the scientific method works, however.

"I've run 10,000 experiments that show toy bricks are green, therefore I can say with great confidence that all toy bricks are green."

Knowing things with absolute certainty is super hard.

The point is not that you can't trust your results, it's that you can't extrapolate a theory about all toy bricks unless you know you are seeing a representative sample of all bricks.

"I've run 10,000 experiments that show toy bricks I found on the ground are green, therefore I can say with great confidence that all toy bricks are green." What about bricks in the toy chest? What if someone was building something green before you tested?

As this applies to Facebook's statement, all they can assert is that they've found a way to detect fake Facebook accounts that they've tested against the accounts they've been able to ascertain are fake. What about the fake accounts they could not confirm were fake, or that weren't identified in any way?

I have no idea what statement you're referring to in particular since it's not in the article and you haven't cited anything.

Facebook is looking at tackling farming. In that context, the vast majority of accounts are likely to be operated in the same way. They need to avoid detection and they need to do it as efficiently as possible in order to make the most profit. Facebook has now cracked this particular code and an arms race is likely to ensue. But at the moment Facebook discovered this, it was unlikely there were a significant number of farming operations doing things any differently.

No one is claiming this applies to all fake accounts (or if they are, it's definitely not in the article). Obviously a catfishing account will be run quite differently than a farming account.

I referring to the title of the submission (notable because it's not the title of the article, which is against HN guidelines), which is currently "Fake Facebook accounts can be detected by when they post and the vocabulary used". I assume the top-level comment was referring to that as well. My comment up-thread was really mean to address that while it's true that knowing things with absolute certainty is hard, that shouldn't excuse us from making sure our claims aren't overly broad given the experiment and results.
You should read some Karl Popper (Science as Falsification).

You're essentially getting at why science is both inductive and deductive. Sure this article is inductive, but it's conclusion is falsifiable, so people can test the hypothesis over and over again.

Pointing out that they haven't tested the everything in the universe, including the celestial teapot, doesn't mean that it's false. It's just that it has never been falsified yet.

I wasn't making a statement that it was false, just supporting the original comment's assertion that the submission title (which has since changed to match the article title) was overly broad in its assertion (at least that's how I interpreted the top-level comment). Thanks for the recommendation though, I'll take a look. :)

1: The original title was "Fake Facebook accounts can be detected by when they post and the vocabulary used", which without additional qualification I think is quite a strong statement, and has some problems because of that.

It's worth mentioning that not even the headline says all fake Facebook accounts can be detected.....
I think the point is that they don't have a definitive list of fake accounts, so the only thing they've been able to test are accounts they previously knew were fake, or new accounts it flagged that they were able to determine where fake. That leaves out fake accounts that were newly flagged but weren't able to be determined as fake, and fake accounts that are still entirely unknown.

At most you can say some fake Facebook accounts can be detected by when they post and the vocabulary used, which is what I think the top level comment was trying to note.

I agree. I wrote chat bots before that combined peer activity with their actual triggers to determine when it's appropriate to post, that went undetected for weeks at a time.

I imagine this to be even easier on Facebook, as it's not realtime.

Is it in Facebook's best interest to find and delete fake accounts? Fake accounts clicking on ads makes Facebook money (maybe not in the long run).
I would think yes. If they didn't, why would an advertiser spend money with them, knowing full well that part of it was being thrown away on fakes?
It could just as easily be said that it's in Facebook's interest to appear to want to find and delete fake accounts. As long as their customers (advertisers) believe click fraud is being addressed, Facebook wins.
If that were true Google would not have banned fake Adsense clicker. I have experience in FB market and I can tell you that none of us care about Likes. We care about eventual conversion (What % of people called up to know about our product?) which is a very good filter for all fake people.
Product manager at fraud detection company Simility here. I'm very surprised Facebook hasn't put more effort into curbing fake accounts, makes me think it's very low priority for them. We have social network customers who are much smaller than FB, yet have gotten their fake account rates far below FB's.

One effective strategy we've employed not mentioned here is category mapping: if an account of type A, only targets accounts of type B for likes (especially if they ignore categories C, D, etc.), this is usually a high indicator of fraud. For example, one very common strategy is to create a fake account for an attractive female to friend many male accounts (especially relatively new accounts unaware of these tactic). This can be easily detected by analyzing the gender and account age of all targets and coming up with a diversity score. Low diversity score = likely fraudster.

> We have social network customers who are much smaller than FB, yet have gotten their fake account rates far below FB's.

The incentive to make fake accounts on Facebook is orders of magnitude greater than almost any other social network.

> One effective strategy we've employed not mentioned here is category mapping: if an account of type A, only targets accounts of type B for likes (especially if they ignore categories C, D, etc.), this is usually a high indicator of fraud. For example, one very common strategy is to create a fake account for an attractive female to friend many male accounts (especially relatively new accounts unaware of these tactic). This can be easily detected by analyzing the gender and account age of all targets and coming up with a diversity score. Low diversity score = likely fraudster.

Facebook has methods that radically exceed this method in both complexity, precision, and recall.

> The incentive to make fake accounts on Facebook is orders of magnitude greater than almost any other social network.

Not true. In the article, the writer pays Russell $15 for 1,000 likes. Being generous and assuming each of Russell's fake accounts can farm out 100 fake likes, he's making $1.50 per fake account before it gets shut down. Compare that to social networks where you can directly extract payments from other members by listing fake items for sale, laundering payments from fake credit cards (on other fake profiles) to yourself, or link-baiting other users. A single successful fake account on those networks can easily net you $100.

> Facebook has methods that radically exceed this method in both complexity, precision, and recall.

Agreed, and indeed Simility's models have much more complex methods too, but a) I wanted to post an interesting example everyone here would understand and b) I still say Facebook is not using anywhere near its full ability to stop these fake profiles given how rampant this fraud scheme is on their platform. (Again, follow the money, FB has very little incentive to stop these fraudsters who are only inflating their own numbers. It's important to keep them in check, but there's no incentive to waste resources stopping them.)

> Compare that to social networks where you can directly extract payments from other members by listing fake items for sale, laundering payments from fake credit cards (on other fake profiles) to yourself, or link-baiting other users. A single successful fake account on those networks can easily net you $100.

You're comparing apples to oranges: fake accounts used for "like spam", etc. are different with regard to their complexity and scalability than accounts used for phishing. There are phishing accounts on Facebook as well.

> Again, follow the money, FB has very little incentive to stop these fraudsters who are only inflating their own numbers.

Facebook has a massive incentive to stop fake accounts: fake accounts decrease meaningful conversions that lower the ROI for advertisers, which is tracked carefully both by Facebook and advertisers. This directly lowers the price for ad space on Facebook, and makes Facebook look noisier and less impactful than other channels.

Following the money leads a direct, unmistakeable path to a strong incentive to shut down fake accounts.

It's also very bad to accidentally shut down real accounts, especially in cases where users could be confused enough not to return.

> Following the money leads a direct, unmistakable path to a strong incentive to shut down fake accounts.

I think the difference in opinion on that is that if you look at the long term, which you hope FB are, then yes fake accounts that reduce ROI for advertisers are bad. Unfortunately, they lead to a short term increase in FB ad revenue, which disincentivizes stopping fake accounts too effectively, as it may actually be a noticeable dip in revenue, depending on the scope of the problem.

In a worst case scenario, FB might be in a situation where 20% of ad revenue is from bad impressions, and completely stopping that, if they had the power, would have major negative repercussions for the company. There would need to be some hard choice made about the best path out of that situation. Not that I think this is necessarily the case, but it is an example of how the incentives may not be as clear as they seem.

> Facebook has a massive incentive to stop fake accounts

As an advertiser, I am pretty certain this is not the quite case in (current) reality. A large part of FB's proposition for more money from us includes:

A) Pay more for increased reach and engagement.

B) Our traffic isn't decreasing (despite outside reports/indications to the contrary) and you would be missing a massive and engaged audience if you didn't spend with FB.

This combined with the fact that a _lot_ of ad-spend isn't directly attributable to conversions (often by design), means that more "activity" whether its fake or not, drives up ad-revenue for FB.

You see the same issue occur with other publishers by the way. -It is not uncommon for a publisher (or other related party) to purchase a swarm of fake bot traffic to boost impression and engagement numbers of an ad buy they've sold. -Advertisers un-aware of how much of the traffic to their ads are bots vs legitimate humans (read: "publishers stealing money from advertisers") is a major problem for advertisers, but the bigger the publisher, the harder it is to 'not' be on their platform too. (and FB is _very_ big)

> B) Our traffic isn't decreasing (despite outside reports/indications to the contrary)

It's not. Even the "leaks" make clear that overall traffic is still increasing, both overall and per person.

> This combined with the fact that a _lot_ of ad-spend isn't directly attributable to conversions

I've seen direct reports from advertisers at my last job (doing social media analytics) that show how well they can quantify ROI for ad spend. Fake accounts would negatively impact this number, and it would be extremely obvious immediately.

True. It is like saying Windows has far more viruses than Linux and MacOS but often it is because of incentives and market share rather than lack of efforts on parts of MS to curb viruses.

But I don't think FB has put lot of efforts. I was being targetted by some fake account which was a Facebook profile of a company (created as user). I complained and reported the user several times. Facebook has not taken any action. From what I can see a simple regex on name should tell that "Taylor Swift Lover Group Admin" is not a human being and cant have a facebook account.

Facebook generates money from ad views, even if they're fake accounts. There's very little incentive for Facebook to mass remove fake accounts.
Didn't Friendster spend an inordinate amount of effort on detecting fraud accounts?
you're doing the lords work... hitler used the same diversity scores on the jews.
Hi. I worked on Facebook's anti-abuse infrastructure for awhile (I'm still at Facebook, but working on different things now). So while I didn't personally fight spam/fake accounts, I worked closely with those who did. I'll be blunt: based on this and your other comments, you don't know what you're talking about.

I'll go a step further and give you some unsolicited advice. The anti-abuse community amongst internet/game/tech companies is actually fairly close knit since it's one of the few places where everyone is on the same side and lots of "secrets" are shared (including, even, at the spam fighting conference we organized last year). I would bet a lot of people just rolled their eyes while learning of your company for the first time. You're already entangled in one argument from someone calling you on this silliness, but I assure they're not alone. I'd probably suggest reconsidering this approach.

Wow, what a statement from someone so close to it. Not sure how to look at it.

We recently bought some likes for a page via FBs internal system. The likes we eventually received were nearly 100% identical in terms of names, looks (mostly arabian or oriental), even though the region we targeted was within central Europe - and lot's of obvious fake accounts in there.

I used their Pixel + Create an Audience tool to target a Page Like campaign at people who have visited my businesses' website previously. Very low spam / fake account % on that campaign.
Indeed I know it's a close-knit community. Most of our 20-person team came from anti-fraud teams at Google. I'm guessing the "silliness" you're referring to is the talk of Facebook not being incentivized to block spammers. I think kbenson articulated best what I was trying to say, that there are tradeoffs in blocking good users and decreasing apparent user volume when fighting fraud. Facebook would obviously not be wise to catch every single fraudster because there would be a high number of false positives, so a balance must be struck. As I'm sure you know, fraud teams at many companies often clash with the marketing team because they're protecting the bottom line (sometimes at the expense of the top line) respectively, and vice versa.
I worked at a company with a spam variable in the backend. 0 for eliminating most spam engagement actions like likes. 1 for letting all spam in.

We didn't set it to 0.

There's sometimes positive value in spam. Ex Instagram users get a boost when their pictures are liked, by someone real or not.

I'll be blunt, too. I'm interested to know why it's so easy for Facebook users like me to spot fake accounts and report them, while your crack team at Facebook constantly ignores them and allows them to continue proliferating. I'm guessing you didn't get an inside look at Facebook's accounting that disincentivizes removing these fake profiles. Or do you have a better reason that Facebook repeatedly ignores reports of obviously fake users?
I'm curious: do you think Facebook's anti-fraud measures are effective?
For big companies that benefit from being able to say they have lots of users there is a big incentive not to be good at finding fraudulent accounts. I worked in Big Data Analytics at one of these companies $10B+. We were separate from the fraud department. They'd filter out the fraud accounts and we'd have to re-filter because their behavior was so out of whack it would mess up our analytics. We tried to move our filters upstreams and teach the fraud dept how to identify these accounts but absolutely no-one was interested. Also it's common to have bonuses tied to user numbers.
I know Facebook delete a bunch of fake accounts about 9 months ago.
I've paid for fake likes on my FB pages, the reason was no-one liked them when no-one else did, so it was just to get the ball rolling. As far as I could tell they all seemed to be genuine accounts in Turkey.
i have over 100 fake FB accounts... none have ever been detected or deactivated. i don't post anything on them... just use them to log in to stupid things that require facebook integration. i don't have a personal facebook account.

you are all idiots.

My stupid question.

If i changed the language of my fake accounts in response to this, would they perform better because they now no longer work in your filter?

And yet 50% of my Instagram interactions seem to be spam
go and read some of the affiliate marketing forums. Instagram is being abused more than you would guess. People are talking about getting a million followers on there etc and selling shutouts. I barely use Instagram anymore, and I enjoy photography.
Can't you just... only follow people you know and like? (other than the ads, of course). I use instagram sporadically, and can't really say I have any problems with the abuse mentioned.
The spam takes the form of people commenting generic words on all your posts, and their profiles are full of spam.
I've had to go out of my way to tell several of my friends now that their account is liking spam posts. The worst part is they don't know it's happening, since they don't see posts their account is liking.

The posts getting liked are from fan pages with <5 likes, yet the post gets 10,000 likes within minutes. Is there some reason that isn't easily detectable?

Fake likes is not a problem as long the Fake v/s Genuine ratio is stable. It automatically gets factored-in in the cost per thousand impressions for marketeers. It is pretty much like emails. If you get 1% CTR to double your conversions you need to send twice the email volume.

However if that ratio is not stable then I think it will be a serious problem for marketeers because we would not have any metric to determine how much budget we should scale.

This is the reason I suspect FB is going slow on the killing the fake likes. Their strategy will manifest over much longer period than usual.

Facebook in many ways want to have and benefit from fake accounts and fake engagement.

They're not really selling advertising, they're selling KPIs. Facebook is the best platform because they deliver the best KPIs and so get a larger proportion of ad spend.

Think about this: Company spends $X on digital branding campaign, then to make sure they can justify it to bosses etc. they spend $Y to get the views to go with the spend. Did customers benefit from or like the campaign? That's besides the point. Companies essentially pay people to watch their videos (AutoPlay in their FB feed).