307 comments

[ 2.4 ms ] story [ 276 ms ] thread
(comment deleted)
Wow, that's a very stiff penalty if they're not even turning up for their own brand name. It will likely cost them millions. It's a clear warning to all sites out there to avoid similar tactics.
I feel for them. At the end of the day business is suffering. However, a high profile example of the consequences of using these myopic tactics is also good.
They're aren't currently monetizing their site at all, no ads, so how will it cost them millions? The VC's already know that the founders are a bunch of fruitcakes. I think they'll be right back where they left off after the penalty is lifted, probably in 30 days. When the same thing happens to other small websites than those websites are literally destroyed, no 30 day penalties, banned for life. Shut down your website and start again.
Just did a quick "bieber heartbreaker lyrics" query and couldn't find them in the first 10 pages of results...

Outch, this is going to hurt them.

Excellent. They deserved some coal in their stockings.

>If you go to Google and search for [rap genius], rapgenius.com will not be found on the first page

That's pretty crazy. Whenever I put up an obscure site, I notice ranking on the domain happens almost immediately with no effort.

[edit] Holy shit, I didn't realize quite how awful this is. Very often, I search for "<song name> rapgenius" because I really enjoy the annotations; that's not even on the first page of results. Making it worse, the first page (even when searching for a song) contains stories about them making rape jokes and spamming. https://www.google.com/search?q=today+was+a+good+day+rapgeni...

5th page for "rapgenius"

There are two lessons here: (1) don't do shady shit, (2) Google should only be one of many sources of traffic.

For lyrics sites, Google is the alpha and the omega. There are no other traffic sources for that niche.
Wow! Just checked and they ranked 59th for "rap genius" and 51st for "rapgenius" on google.
"<song name> site:rapgenius.com" should help.
The penalty that has been applied will degrade heavily with time. Rap Genius will have an opportunity to rebuild their ranking in the coming months. I suspect they'll be fine, as the product is solid; so clean-up in aisle three, and then business as usual. They're also very well capitalized to withstand this.
12/25/2013

And the greatest gift of all was to be schadenfreude.

For people who act like arrogant jerks were punished severely that day.

Why did not they penalize their competitor websites which are using the same (even worse) tactics?
>> "Why did not they penalize their competitor websites which are using the same (even worse) tactics?"

Source? Proof? RG was penalised because their method were made public and Google was made aware of it. If there is proof other sites are doing the same (I don't doubt they are but proof is needed) I'm sure Google will penalise them too.

As I remember, In the "apology" letter RG provided a list of resources that prove that other websites are doing the same thing.
What they provided wasn't exactly 'proof'. Without any conclusive ground they accused other lyrics sites. Linking to Open Site Explorer and saying they they think some links might be suspicious hardly counts as proof, IMO. Besides, now that the matter has been brought under Google's attention, they'll of course penalize the other lyric sites too, if any violation is found. But in case of RG, it was all out in the open and I think that's why action was quick.
Their "proof" was a load of crap. AZLyrics has several lyric websites for specific genres (punk, metal, and so on) which are actually really good if you're a fan of that specific genre. Of course these sites link to each other because they're a part of the same company. It isn't anything like what RG is doing.
Its very scary to build your business on being ranked high on Google. When you build your house on sand...
More like... when your entire business is built on google rankings maybe you should consider reading their rules. Closely.

This is a case of Rap Genius brazenly breaking the rules and thinking they can get away with it because of their gilded connections (their spoiled frat boy persona seems to reflect the reality of their behaviour: thinking they're too rich and connected for the rules to apply).

Acting like a clown, like Donald Trump, definitely gets you money and exposure. But personally I wouldn't take any amount of money if it meant I became a person making public rape jokes, making startup culture even more sexist, regressive, pathetic... not to mention racist. Their tongue-in-cheek appropriation of black english is mocking both the rap community as well as the poor black AND white communities that actually use that language for serious (and have no chance of going to elite schools let alone getting a half decent education).

Anyway Rap Genius is a nice service (and I'm a fan of hip hop) but the behavior of the founders is disgusting. Shameful behavior does not suddenly become cool just because it can make you a hundred million dollars. Capitalists always sell away their values eventually. It's sick.

Know what's cooler than a billion dollars? Being a decent human being.

I think this could be a deathblow to Rap Genius.

I performed the following searches in an incognito tab and could not find rapgenius in the first two pages of results (I did not look beyond the first two pages):

"rap god eminem lyrics"

"machine gun funk rap lyrics"

"lose yourself eminem lyrics"

it looks like they might have stopped showing "rapgenius.com" in non site-specific results. Searching for "rap genius eminem lose yourself" doesn't give any results in the top 5 pages. The only time I can see results from them is by adding a "site:" modifier to the search.

I think you're right, that depending on how long this lasts, they could be in a world of hurt.

Damn, I tried searching up some rap lyrics that I knew worked a couple of days ago, no dice. How long does this penalisation last?
The sad part is that RG was the most high quality lyrics site I've seen. Their revenue is going to take a huge dive, much more than their link scheming was hurting Google (or probably anyone else for that matter).
It doesn't matter that they had the best experience and content because Google was forced to do this to set an example and show that they aren't playing favorites. Imagine other companies who will start testing more black hat strategies with the expectation that it's okay as long as they believe they're better than sites ranking higher than them.
You know what? If they actually are better, then that's only a good thing in my opinion. Who cares about "fairness" here, we want the best results. That's one way of looking at it anyway.
I agree it was what they should've done. At the very least it was the right direction. Hopefully their reputation is enough to keep them afloat while they build up again.
I suspect the several million in funding they've taken can also keep them afloat for a little while.
Its also sad thay they went about it in such a stupid, stupid way.

They could have built a nice "Link to your favorite lyrics" widget for people to include in their "homepages". Make the appearance customizable, jumble the HTML, and include some stock bla bla to make it "individual" and to appeal to google.

Then announce a competition. We'll showcase the best homepages that link to our lyrics in the most creative way. Maybe give away some silly prizes. Completely legal, nothing for Google to complain about, but gives you the same effect very cleverly.

Instead they did this brazen crap... makes them look very amateurish. Technically not very different, but the tone is completely different, and Google had no choice but to penalize it.

Right on. I'd use a widget to embed lyrics with audio in every blog post if they made it easy. May be integrate with spotify. And create a Wordpress plugin.
Where can I find the guidelines that specify what is legal under Google rules and what is not? As someone who isn't familiar with rules, it's not obvious to me what the technical difference is between what they did and what you're suggesting. On a gut level your suggestion seems less spammy, but I'd like to know what the actual rules are.
Agreed. It is natural for a website to want to get traffic. How can Google prove that RG's tactic was for SERPs and not just for the direct link traffic from blogs?
That might have worked a few years ago links via widgets like that high risk I am afraid
agreed, one of the better sites out there but what revenue?
(comment deleted)
Tried all of these just now:

  Justin Bieber All Bad Lyrics
  Justin Bieber Confident Lyric
  Justin Bieber Heartbreaker Lyrics
  Justin Bieber Memphis Lyrics
  Justin Bieber One Life Lyrics
  Justin Bieber All That Matters Lyrics
  Justin Bieber Hold Tight Lyrics
  Justin Bieber Pyd Lyrics
  Justin Bieber Change Me Lyrics
  Justin Bieber Recovery Lyrics
  Justin Bieber Bad Day Lyrics
  Justin Bieber Roller Coaster Lyrics
  Justin Bieber Lyrics
All of these just yesterday were in fact ranked in the upper 5 (very often #1 actually). Rapgenius results are now not even in top 10.

Most surprisingly, even

  Justin Bieber Heartbreaker rap genius
will not show up on Google results. Wow, that was a swift and harsh response. The only surefire way to get to a RG site is by doing "*bieber site:rapgenius.com".

They were using a wide array of questionable SERP optimizing techniques (http://www.rocketmill.co.uk/hideous-seo-strategy-rap-genius among others). One described here is very interesting: https://news.ycombinator.com/item?id=6958883. Apparently that's an old one, but it's first I'm hearing of it. From all of this, to their rape jokes and everything else in-between, you can't say they didn't have something like this coming.

I liked their rape jokes. Grow a sense of humour.
Quite frankly not showing me rapgenius results when that's one of the search terms is idiotic. I guess now I have to remember to go straight to their site whenever I'm looking up lyrics. I'm glad I at least know about them and I feel bad for the people that are now going to be trapped in craplyrics.com hell.

I understand and agree with google's desire to punish spammers but their effort should not have such a major impact on users who obviously know what they are looking for.

Just wow. I think the intention was to make an example of Rap Genius for other companies. I honestly had no idea how severe these violations were until now.
> I think the intention was to make an example of Rap Genius for other companies.

Doubt they intended to penalize them quite this drastically. Its likely that they algorithm is just being overzealous some keywords being blacklisted. Quite similar to how the UK porn filters are filtering out Claire Perry, the most prominent campaigner for the filters[0].

> I honestly had no idea how severe these violations were until now

The violations are pretty serious and have been for the better part of the last (half-)decade. RapGenius should have known better. For me the worst was when the pretty much said "But everyone else is doing it".

Too bad this will obliterate them.

Unintentionally though, going forward RapGenius will be the poster child for what happens to children on the naughty list.

[0] http://www.forbes.com/sites/timworstall/2013/12/25/the-best-...

I agree, they are clearly trying to make an example for others.
Just use Bing for now :)
Even Googling just "rap genius" doesn't get me their site in the top 10 (just their Twitter).
Now, what about actually getting quality site when I search for "Justin Bieber All Bad Lyrics"? The first page now is now just full of low quality lyric sites.
Well, really, are they? I have Adblock, and really the sites I get from "Justin Bieber All Bad Lyrics" are fine if I was just interested in the lyrics of that particular song.
Shouldn't "Justin Bieber All Bad Lyrics" just show you all Justin Bieber lyrics?
We are all much better off now that azlyrics.com is back to its rightful place at the top of the lyric search results.

Rap Genius alleges that their competitors are using the same tactics. Will those sites also get the same scrutiny? Perhaps now would be a good time for a new lyric site run by some nice bland people to go on the offensive and rise to the top.

So, the only reason RG in the first place was able to provide lyrics with no spam at no costs is because they were riding on VC money. Other lyrics sites have to make money somehow. But then you might say -- well, what does that matter to the audience? The answer is, usually the spam-free groovy train usually does eventually come to an end. Sites can go on for long periods of time serving straight up spam/ad-free content, but that always ends. Who knew what RG was intending to do after this ended? Would they have gone the Facebook route of datamining the hell out of Biebers fan to advertise them shit they don't need?

One last thing to keep in mind: Rapgenius was an unlicensed lyric site. They weren't playing ball on either side of the fence. Let's hope that a lyric site emerges that provides lyrics with at least acceptable, non-distracting ads, and serves licensed content. I really think merely going by this strategy will get you ranked high on Google.

It makes sense that a site needs business model to be sustainable, but Rap Genius is still searching for or developing their model. It does not appear to be based on displaying ads alongside lyrics, and that also is not the main point of their site.

Rap Genius was originally called Rap Exegesis, and while it has a lot of rap lyrics it is a community supported text annotation site, with an ambition far beyond a lyric repository. They have a great deal of content beyond rap, or even lyrics in general. They have poetry and literature, announcements, and press releases (I'd love to see them make a Rap Genius page of Ulysses with the contents of Ulysses Annotated). They have a platform and are building a community. That's why it's attractive to VCs, and that's why it's something beyond the eyesores one mostly lands on searching for lyrics.

I hate that lyrics have to be licensed for informational purposes. I think it's stupid.
writers (a.k.a the anonymous person who wrote your favorite songs) make their living writing those lyrics & deserve to be paid for their work
No one deserves to be paid simply for working. You might be the best X in the world but if the world doesn't value X then you don't get paid. Pay is earned by doing things people want to pay for, not simply by doing things. If people don't want to pay for music/lyrics then music/lyrics has no market value. You don't deserve anything.
Clearly there is value since there is demand. What you're saying essentially is that peoples right to pay what they want outweighs the creators right to set their own price for their creations. That's not the way the world works. Demand can dictate price to an extent in that the seller wants to maximize earnings but it doesn't dictate it absolutely.

These companies(RG, azlyrics, etc) are making money where their main product is the creative work of someone else. Damn straight that other person deserves a cut of the profits because without them there is no site.

> What you're saying essentially is that peoples right to pay what they want outweighs the creators right to set their own price for their creations. That's not the way the world works.

No, that's exactly how the world works for luxury items like music; demand drives, not supply. What the seller wants to charge means absolutely nothing if demand is elastic and consumers can simply stop buying. Demand sets the price, not the seller. Creators can set whatever price they like, and consumers can and will ignore them; if you want to sell something you have to set a price the market will actually bear.

> Damn straight that other person deserves a cut of the profits because without them there is no site.

No they don't; they got paid to write the song, they're done. The notion that a person deserves to be paid forever in secondary markets where people trade their creations is a perverse and absurd notion that exists only in the messed up modern world; it's wrong and it's stupid. You as an artist do not deserve to be paid every time someone makes a copy of your art; period. If I paint a copy of a Picasso, I don't owe the Picasso family anything and the same should apply to any other creation. Copies are not theft and the artist doesn't deserve anything.

The seller sets the price. People either don't buy it or acquire it without paying. But the buyers don't set the price. In this case the buyer is the website and they have to pay the price since they are using the product openly and publicly.

>No they don't; they got paid to write the song, they're done. The notion that a person deserves to be paid forever in secondary markets where people trade their creations is a perverse and absurd notion that exists only in the messed up modern world; it's wrong.

Who is this magically being that pays the writer to write the song exactly? It isn't a secondary market it's a primary market. The song writer has created something(A song!) and the site wants to use that song. They are dealing with the song writer or copyright holder that is a first sale.

That person deserves to be paid for what they have created which is what this is. Your job wouldn't exist without that belief and neither would mine.

> But the buyers don't set the price.

I never said they did, please do try and comprehend before simply repeating yourself.

> That person deserves to be paid for what they have created which is what this is.

False.

> Your job wouldn't exist without that belief and neither would mine.

This is a belief you hold, not a fact.

> No they don't; they got paid to write the song, they're done. The notion that a person deserves to be paid forever in secondary markets where people trade their creations is a perverse and absurd notion that exists only in the messed up modern world; it's wrong and it's stupid.

Whether or not a person deserves to get paid forever for their work is irrelevant to this discussion... The point is that someone is getting paid forever in this situation -- either the original songwriter or the owners of the lyrics site. I'll accept for the sake of argument that it shouldn't be possible for a person to monetize his or her work in perpetuity -- so then why the hell should it be allowable for an unrelated third party to do so?

> I'll accept for the sake of argument that it shouldn't be possible for a person to monetize his or her work in perpetuity -- so then why the hell should it be allowable for an unrelated third party to do so?

Ok, fair question. I didn't claim a person, be it first or third party, shouldn't be able to monetize anything; I said they don't deserve it, not that they can't do it. I'm disputing the sense of entitlement displayed by the use of the word deserve. No one, whether the creator nor the third party, is entitled to consumers money.

So it's ok to copy and resell books without compensating the author?
> they got paid to write the song, they're done.

In the vast majority of cases this is wrong. Most songwriters do not get paid to write songs. They get paid only when they receive royalties on the reproduction of their song.

This is good for society because it means that the songs that everyone likes will provide the most reward--so songwriters have a strong incentive to write the best songs they can.

If someone makes money off a copyrighted work, the copyright holder deserves to be paid.
No they don't; they might legally be required to pay, but legal != deserves. Copyright and patent laws are wrong and create artificial scarcity to create a monopoly where there should be none. Just because something is the law does not make it just or right; they don't deserve anything.
So you would do away with all non-physical trade?
Where did I say or imply this?
"Copies are not theft and the artist doesn't deserve anything."
That doesn't imply what you seem to think it implies.
Then clarify because so far all you've done is said "People don't deserve this! Copying isn't stealing. They shouldn't get paid for this" then when anyone challenges you they get the reply "Thats not what I meant" or some snarky comment about their comprehension.
Clarify what. I could send you a bitcoin, look, non-phisical trade. If you want to critique something, it's up to you to make a clear critique, not me to figure out what you mean when the words you wrote are trivially and obviously false. When you infer I mean things I didn't say, it's not snarky to point out I didn't say that or mean that. I said what I meant, deciphering meaning was not an exercise left for the reader, it's right there in the words I wrote.
You're wrong. Saying that music/lyrics has no market value is just as stupid as saying that just because I personally don't want to pay for a Mercedes Benz means it has no value, and I can take one freely.

Clearly SOME people are willing to pay for the right to use lyrics, ie. singers, record labels, etc. Just because a single individual, ex. rapgenius, doesn't want to pay for the music/lyrics, which they legally are obliged to do, doesn't mean that music/lyrics have no market value. There is a market value which has been established, and which people pay for. It just means that if they don't pay for the lyrics, they are stealing it.

> You're wrong. Saying that music/lyrics has no market value

Well since I didn't say that, you're wrong.

You're right -- no one deserves to get paid simply for working (i.e. creating a work). But everyone deserves the opportunity to sell that work. When someone else takes what someone does, it diminishes the value of that opportunity. Copyrights protect that opportunity.
> But everyone deserves the opportunity to sell that work.

We agree.

> When someone else takes what someone does, it diminishes the value of that opportunity.

No it doesn't, those people wouldn't have paid anyway. Copying is not theft. This is the same tired argument the MPAA's been trying to make and it's just a sign of the establishment not adjusting to the new reality that distribution is not a profitable industry anymore.

> No it doesn't, those people wouldn't have paid anyway.

You can't know all of the ways they will possibly make money from the content. Justin Bieber may want to charge Apple more for including vetted lyrics in iTunes. But why would Apple give Justin Bieber money when they can just copy the content from Rap Genius? Or cut a deal with Rap Genius for a feed?

> distribution is not a profitable industry anymore

This is where you're missing the point. You see the price as paying for distribution, but it's really about the value of the content. Historically, the two were inextricably linked by a physical medium. Now they are completely separate things with different prices and costs, and are almost unrelated to one another. Just because one goes close to zero, doesn't mean the other does, too.

Nor do I care about the ways people like to think up to ban other people from doing stuff. If you don't want your work copied, don't publish it. The whole copyright/patent system is a ridiculous absurdity based on the false premise that creative people require monetary incentive to do anything. When technology renders enforcement of laws impractical, then the laws are simply bad. Copying is not theft, creators are not entitled to anything, they deserve no protection for their published works at all.

Art doesn't have to be a business and art will be created regardless of whether artists can profit in perpetuity from their work. The notion that ideas and words are property and can be owned is perverse doublespeak meant to trick people into transferring their gut instincts about physical property onto things that quite simply are not property.

Performers should make money from performing, not being paid forever because the government created artificial monopolies on words and ideas.

Are you for real? The world values the lyrics/songs, and are searching for them. There is a system in place to compensate artists, and "OMG DISRUPTION PIGGYBACK" is getting really tired.

Commenting on established artworks with a cute little javascript popup doesn't add anything to the original art. If I want to check lyrics, I am looking for the lyrics, not a popup that shows a picture of a ham when I hover over 'ham'. RG doesn't even have good content, just a bunch of smartass nonsense and obvious comments.

> The world values the lyrics/songs

Never said they didn't, try understanding what was sad before flying off the handle.

Are you saying that songwriters deserve to be paid whenever I simply want to know what the lyrics of a song are? That doesn't make any sense to me.
Are you saying writers of books deserve to be paid whenever I simply want to know what their book says?
Could you provide an example that is even more unrelated? This one is still about writing...

(Seriously though, this is something absolutely different)

Its exactly the same, just a matter of length of the content.
Are you saying writers of poetry deserve to be paid whenever I simply want to know what their poem says?

Is that close enough for you to see the similarity, or do I need to explain how songs and poems are functionally indistinguishable and further how poetry and prose is functionally indistinguishable; and therefore how written works are functionally indistinguishable and if you support copyright in one but not the other your perspective is logically untenable?

Or perhaps you'd like to explain to me why a poem should lose its written copyright rights when someone sings it.

Songs are (in general) not "musicized" poetry. It's closer to a movie. People work on it and it sells as a whole. Lyrics + music + singing or screenplay + scene + acting (simplified). I don't see why I should pay anyone for quoting a movie. Nor I don't thing lyrics of a song are worth anything as standalone. Those who wrote the screenplay/lyrics have deal with owner of the final product and are paid from that (or were paid once and that is one extra reason to not pay them over and over).
(comment deleted)
But what about the links on the left titled "Recommended Sites", they list pretty much identical websites.
(comment deleted)
I wonder if Rap Genius thinks their scheme was worth it in retrospect.
That's the exact same logic as victim blaming.

The person who posted it has no obligation to white wash YC funded companies.

Very bizarre line of thinking when you put blame on the HN poster and not Rap Genius for doing all kinds of black hat SEO.
It's obviously wrong what Rap Genius did, no doubt about it. They should not have used these techniques. But, it's not a bizarre line of thinking to think of whether the person who posted it finds it to be worth it after the fact given the severity of the repercussion (which could effectively ruin the whole business).
RG's techniques were not honest and were costing other sites that were using legitimate techniques and playing by the rules their ranking in Google's search results (and maybe even revenue).
mlyang: It is absolutely a bizarre first reaction to imply that the reporter should feel culpable for the harm that RapGenius caused themselves.

If RapGenius is ruined that easily, RG didn't have a real business to begin with. Their scheme was something that would've certainly triggered Google's ban-hammer sooner or later; the only question was timing. My bet is that RG will reexamine and improve their distribution playbook. I say this because I don't think a16z would invest in a company whose distribution plans consisted solely of SEO.

But you could be correct. Maybe RG is such a fragile company that this nearly inevitable setback is enough to kill them.

p.s. just an FYI: it's generally considered bad form to delete a post that has already garnered responses, especially if those responses are critical.

I never implied that the reporter was culpable. He isn't, RapGenius 100% is. But if I were in the original poster's shoes-- for me at least-- I would've initially found breaking the story to be fun and interesting, but seeing how Rap Genius as a business is basically screwed for the foreseeable future, I would've definitely thought twice. I agree that Rap Genius is 100% wrong in its shady tactics, but as a lot of the other commenters in this above thread (who have not been down-voted to oblivion) have stated, this is a pretty harsh punishment on Google's part. Knowing the harshness of the punishment after the fact, I was just pondering as to how the original poster felt, that's all.
Perhaps you didn't intend to imply it, but unfortunately that message was communicated to several of us. Thank you for clarifying that you don't hold that view. This apparent miscommunication emphasizes the importance of not deleting (and not substantially editing) posts once they've garnered attention.

I, for one, hope the author feels no remorse for their article. It was a fun and interesting article, and they reported the story well.

I continue to hold the belief that if this incident creates a material problem for RG, then RG was already a ticking time bomb. That said, I have plenty of faith that RG will weather the storm, and that perhaps this will be a wake-up call that inspires them to do something more creative and valuable than a spammy link-swap.

The person that posted it should be a complete opportunist about the matter and put it on their resume and apply to Google or something.It would be even funnier if they offered publicly to be a paid consultant for rap genius.
I think the best thank

you Would be to outflank

google And fill the blanks

With lyrics that lank

them back out of the dunk-tank;

Justin Bieber lyrics CLICK HERE

Classic blaming of the messenger.
Let's not feel sorry for these guys just because they were part of YC. They were taking part in black-hat practices, which were clearly in breach of Google's ToS. I expect this penalty to expire eventually, but let this be a lesson to anyone else thinking about doing the same.
I'd like to see if they penalize other lyrics companies similarly and commensurately though...
Report those other companies to Google's anti spam team.
Better yet, post an exposé on HN (that seems to be pretty powerful these days).
(comment deleted)
I'm shocked to see that their previous fairly good ranking has been fraudulent all along.
I don't think it was. I think this is a huge penalty, not a correction.
Disagree, it's the best lyrics site by a country mile. Every other single one of them is dogshit (except OHHLA.com).
Try this:

[bieber heartbreaker lyrics rap genius] or even [bieber heartbreaker lyrics rapgenius.com]

I wonder how many "legitimate" links a site like RapGenius gets. I've never seen anybody link to them, except maybe in social networks. Similar for the stackexchange sites. Sometimes people link to good questions from blogs, but a large portion of links are probably from other stackexchange sites, twitter, and Jeff Atwood's blog (which is a marvel of SEO and affilate marketing in itself).

For many of these high quality content silos (for lack of a better word), the PageRank paradigm seems pretty broken. People don't feel compelled to link to well-known sites precisely because they are well-ranked. And when they do, it is in form of viral posts in social networks, which is not really a great input to determine site quality.

I wouldn't be surprized if Google's real secret algorithm nowadays consists of millions of special cases maintained by thousands of poorly paid interns.

Why poorly paid?
It's the cliche. But you're right, the most menial worker at google earns probably a lot more then I do (in academia). I don't know...
I was thinking something along the same lines. PageRank was great back in the day, but in today's world, the link is not the only way of determining whether a site is good or not. I wonder if a rethink of the HITS model or a complete overhaul of search ranking based on social and reputation cues is in order?

StackOverflow does well despite few people linking to them because the search terms are so niche, if I search for a fairly generic programming term I maybe get one StackOverflow post in the first 10 but if I search for an error message, they'll all be SO. Other StackExchange sites maybe don't benefit quite so well from that long tail, though. Sites like Quora presumably have the same problem.

It's interesting how sites like these, and in turn mobile apps, are turning information into vertical silos (which want you to stay within their bounds) rather than the horizontal web of equality we're used to.

(Also, the interns are pretty well paid, but yes. They start out as autocompleters: http://www.youtube.com/watch?v=blB_X38YSxQ)

The genius of the original PageRank is that links serve as machine actionable artifacts of 'social and reputation cues'.

But of course Google's algorithm is way more complicated now, and takes into account all sorts of things. They of course have all sorts of smart experienced people working on improving it, and apparently still determine that links should count for a lot.

I'd guess there are actually lots of links to stackoverflow.

Are you saying SO questions high google rank is mainly a result of links from within the SE network?
Yes, internal links can be extremely valuable especially in narrowly relevant niches like SO.
No, but I guess e.g. physics.stackexchange.com or judaism.stackexchange.com benefits a lot from the SE internal links, and I guess for those smaller sites it is the main source of page rank.

That is, if page rank still works like Google claims. I'm pretty convinced that there is a lot of manual curation going on. At least for popular terms and for the first page of results.

> and Jeff Atwood's blog (which is a marvel of SEO and affilate marketing in itself).

Really? What makes you claim that?

Can someone explain to me why Google is formalizing this against Rap Genius?

I really like their n-gram viewer of the New York Times wedding announcements. http://blog.visual.ly/rap-genius-new-project-visualizes-30-y...

I would be interested in learning what they have to say about NLP (Natural Language Processing) since they deal with language which is quite vernacular.

I can't imagine that this is permanent. Rap Genius is actually one of the most high quality lyrics sites out there, not another spam farm. So removing them for good actually reduces the quality of Google's search results.

I do suspect that because of the attention this once case received, and since Matt Cutts was personally looking into the company's SEO tactics (talk about bad luck), they issued a public, but temporary, slap. That would serve as an eye opener to Rap Genius about how dangerously they're playing the SEO game and a warning to everyone else tuning in.

If the "slap" is too temporary then it backfires as a deterrent because such a slap would teach future companies that you might as well game the system and then accept your small, temporary punishment if you do happen to get caught, which you might not in the first place if you are more clever about doing it than the rapgenius guys were. There's really no point to Google making public spectacle of a penalty that might end up so temporary as to be toothless.

In any case, I find it hard to shed a tear for these guys, they either knew full well that they were flagrantly disobeying the Google rules or they are incredibly stupid. And unlike some others I'm not so surprised that their previously high ranking might have been gamed because they are one of a set of companies I know of that has huge mindshare with the HN/startup crowd, but virtually none with people I know who aren't part of that first group.

It's not permanent.

Google wants to make a point, they turn off the traffic faucet, Rap Genius repents, and things slowly return back to normal.

They don't even rank for "rap genius" right now. Their Facebook page does though.

You don't think this gives competitors a chance to compete better?
This just makes Google less useful. RG had the best lyrics and annotations for lots of songs. Now Google sends me to barely-readable sites that only exist to show as many ads as possible per page, even if I include "rap genius" in the search.
We should start the same kind of campaign to add these kind of links to the bad lyrics sites. Maybe Google will penalize them too.

Btw., duckduckgo still shows rapgenius on top.

You could always go direct to RG.
(comment deleted)
That's beside the point. Millions of people that search Google for lyrics every month getting sent to worse sites is a problem for Google, not for me.
The disappearance of 1 lyrics site(albeit a good one) from the face of Earth doesn't even come closing to putting a deny into Google's search quality. There are a million other sites out there. RapGenius is the one that has lost, not Google
> This just makes Google less useful.

In the short term. In the long term, this move probably has prevented a lot of black hat SEO and spam from happening on the web.

It's not about whether or not google is useful for finding this particular site and this particular niche, it's about making sure their search results overall aren't being gamed. Sucks for RG (though they pretty much deserved it here) but better for all of us that use search engines to find good content, which is pretty much all of us.
These guys will never grow up
(comment deleted)
That just proves how google search algorithms become inaccurate. I agree, RapGenius did some shady things, BUT it still has a better content then other websites which are now ranked much higher.