HN Explorer

Ask HN: Google crawled our site, and the changes shocked it. We broke up

11 points by geoffw8 ↗ HN
Fellow HN'ers. I feel like I'm dating Google. Every now and then we go out for dates (crawls), and if I order the wrong meal, wear the wrong shirt - she gets pissed.

My problem is we set up a wiki to sit alongside our site, we thought "great! content! Google's gonna love us". So we set up a Mediawiki installation.

Now I guess Google had us down for a certain set of terms (pre-wiki) - it had learned what we do and was sending us a reasonable amount of traffic.

Now, post-wiki Google thinks our most significant word is "Media wiki" - our traffic has dropped right down to 10% of what it was. Essentially, we confused Google as to what we did.

My question is: now we've removed the wiki link from our site, and added the /wiki directory to the robots.txt - when the crawler comes back, will it effectively reverse the changes? Will Google "forget" the old content?

Anyway, I'm glad we learned this now - and not in 6 months time.

Very much appreciated for any insight/wisdom you guys might be able to offer!

7 comments

[ 2.8 ms ] story [ 28.6 ms ] thread
TuxPirate
Bear in mind I am not familiar with media wiki but this sounds like a misconfiguration of the web app.

As you don't provide enough details or even partial statistics about your visitors it's hard to analyse the situation with you.

> Try asking around in #seo on irc.freenode.net

geoffw8
Thanks, I'm going to head there now!
ig1
Are you using Webmaster Tools it might give you a better insight into what went wrong?
geoffw8
Yes, we are. This is what gave me the insight I mention above. Our significant words were "women" and "clothing" (we're a women's clothing shopping site) now they say "media wiki" is our most significant...!

Its what leads me to believe Google has almost chalked us down for the wrong set of terms, given that it must have thought Mediawiki was almost a big part of our site.

Theres that and the Mediawiki duplicate pages, too which doesn't help our cause...

AndyNemmity
I actually had the same thing happen, so I'd like to know the answer if you find it.
mattgratt
If you share your site's URL, I'll take a look. (I know there are a fair number of really good SEO consultants on HN so maybe we can help.)
bjones
The keyterms listed in Webmaster Tools are just a count of their occurrence on your site. We've played with this and can very obviously see our site move up and down in the rankings based on simply changing the number of times certain terms appear across our sites. We have a number of new sites where there probably aren't many other factors going into our ranking, so this is especially apparent.

We've seen these changes take on average 2-6 weeks to get noticed through webmaster tools. We've made many rapid changes to a few of our sites and it has clearly confused google. These few sites have been listing the wrong keyterms for months and we haven't seen any sign of them changing. One site even randomly switches back and forth between listing relevant keyterms and listing latin leftover from our Lorem Ipsum that was accidentally released while the site was being built.

Moral of my story is to not make too many drastic changes to your site, but if you've made a few, just relax and let google run it's nonsense. If it doesn't return to your old content, then I'm not sure what you can do.

The most important thing to learn about Google is that there really isn't much you can learn. Just pay attention to numbers and try to get a "sense" for what is going on. I'm not an SEO or Search expert, but I think experts just have a lot of time in front of the numbers.