HN Explorer

TwitterSpam

15 points by axod ↗ HN
There's been discussion before about spam on twitter, so here's a concrete example.

It seems like people are just looking at the trending hashtags, and spamming, which then gets them into search results. How can twitter solve this? Any ideas?

For example, I did a search for #media140 which is an event in London today. The whole page of results was all spam tweets from the same account, with popular search terms and hashtags in it, (And an affiliate/target link).

eg http://twitter.com/Janet104807/statuses/1859946423

also http://search.twitter.com/search?max_id=1859950510&page=1&q=www.goldencasinoflash.com

Another point is this spam would artificially increase the "trending topics" - feedback loop.

It seems like the search functionality could become useless pretty quickly if spamming isn't stopped in its tracks. It'd be very easy to write an app to create twitter accounts, find trending hashtags, popular keywords, then start sending out tweets with affiliate links. Go through web proxies to avoid any banning by IP. Do they already exist?

Google solved this for website search with PageRank, and other algorithms to detect spam, how could twitter do a similar thing?

Thoughts?

23 comments

[ 2.6 ms ] story [ 61.4 ms ] thread
jgrahamc
1. Make sign up hard so that bots can't do it

2. Block accounts the send out the same URL multiple times

3. Block tweets that contain multiple, popular hashtags

4. Block accounts that send multiple messages over a short period of time with the same hashtag

5. Allow users to flag accounts for review

6. Implement SURBL for URLs

ivankirigin
5. Don't display results chronologically
axod
What other order makes sense for 'realtime search'?
antiismist
Time of course has to be a factor in the sorting of the search results. But that doesn't mean that it has to be the only factor.

Other possible (negative, antispam factors): ratio of following to followers, number of duplicate or near dupe posts, etc.

sfphotoarts
relevance makes sense, where chronology is just one aspect of relevance.
axod
1. This would surely hamper growth for real users. No one wants to verify email or catchpa.

2. Tinyurl could help them, but also there's a million affiliate programs out there, you could paste a different affiliate link each time. Also surely a lot of accounts legitimately contain links from the same domain - eg new blog posts.

3. This example was using hashtags, but also just popular search terms - "american idol". I don't see how you can detect that easily.

4,5. Agreed, those I think could work well

thamer
> 1. Make sign up hard so that bots can't do it

That could be overkill...

> 3, 4. Block tweets that contain multiple, popular hashtags. Block accounts that send multiple messages over a short period of time with the same hashtag.

So, would you block these? “OMG I TOLLY FELT DAT #earthquake #sf”

“Someone at the corner of XXX and YYY? Baby crying under rubble #earthquake #sf”.

Why not a simple spam filter like the ones for email? Google seems to have a pretty good one now - a similar technology could be trained on tweets. Maybe not block, but just increase their spam score? A good filter might even figure these rules by itself.

csomar
Create a Twitter Rank, people are ranked by number of "replies" from users with high rank ;)
sosuke
a TwitterRank would only increase the number of spammers to reply to their own spam to increase their rank artificially just like it's happened with Google. Google has seemed to move away from PageRank as the primary indicator for its searches anyway as you can have page 5 results with the same PageRank as page 1.
csomar
true, but page rank still a key, there's a difference between PR2 and PR7. Spammers can rise up to PR4 their pages, but PR7 or PR8 will be impossible.
sosuke
I just made a similar discover on my own I just posted about this morning. I tried tweeting something like "online dating" and was able to get a follower that was spam in a few minutes. My biggest problem is trying to walk the line between being spam and self promoting your product.

http://news.ycombinator.com/item?id=618580

noodle
simple. implement a solid, clear-cut spam policy and include a button on a tweeter's page to flag the account for review by admins. kinda like we do it here :)
axod
It's different. Twitter search is essentially the [new] view here. Which, if you look at it, does contain lots of spam.

Once someone has seen the spam, the aim has been met. If they close the account, it's not a big deal to open a new one.

noodle
on this level, the level of an individual trying to get a single link seen, if they really want to spam a link, there's very little that can be done to stop them.
csomar
controlling millions of tweet a day is barely impossible
ivankirigin
You can @reply the @spam account to tell them about a spammer, or just block them. Lots of blocks alert the admins too. There are certainly some auto kill features too.
noodle
true, true. but how many of your average users know this? i'm just saying, your options and the process are not common knowledge.

it would go a long way if the whole twitterverse were policing spam instead of just the more savvy users

paulgb
Hiding new accounts (7 days, for example) from appearing in the real time search would be a partial solution.
ashleyw
Wouldn't that just fix the problem for 7 days, to which the bots would use accounts they created the previous week?
paulgb
Quite possibly. But at least it makes spamming less convenient. In my experience, most spammers are lazy, so if you make spamming your site even marginally harder than spamming the somewhere else, they go to someone else. Granted, this might only apply to "long tail" sites and not giants like twitter.

The other advantage is that there is probably a black market for twitter spamming scripts. This might break a few, at least temporarily.

The most determined spammers will probably always find a way through any automated system, but making it inconvenient is good if it is easy enough to do.

siculars
twitter will need to borrow a page from the anti-spam war being waged in email land.

-rate limiting

-de/duplication

-source analysis

and a variety of other anti-spam measures. all this adds to the computational cost for twitter but in the long run is in their interest.

collision
Accounts suspected of spamming with hashtags should still be able to tweet, but their tweets shouldn't appear in the search results. Problem solved?
misuba
Do as Google Accounts does (or did last I checked), and require either a mobile number or a referral to create an account. For Twitter the mobile part even makes sense, sort of.