HN Explorer

Tell HN: New Google homepage, A/B testing gone crazy.

55 points by Loic ↗ HN
Today, I was served the new Google homepage, the first that struck me out was how inconsistent it was. I started to look at all the inconsistencies and found many of them:

- 3 different arrows for the more links. - mixed use of upper/lower case for the "M" in the more links. - the position of "advanced search" between the home page and the results page. - the position of the search button (kind of within the input field on the results page but below with space on the home page). - you see "Everything" with an icon in front of it (why the icon?) and then you have a link with more, what is more than everything? - etc.

It can be a game to find so many design errors, things totally wrong when looking at the whole page.

I asked myself, why? Then I found, it is simply A/B testing gone crazy. They have tested gazillions of independent small changes in their multivariate tests, found the optimal combination, but, forgot the big picture.

So maybe the new home page and results page are the optimal ones, but they are inconsistently optimal. For the first time in many years, the Google pages now "feel wrong".

This is annoying me a lot, for 2 things:

- I hate it when I search. - It will spread like wild fire because if Google has done it, it must be good, so bad designers will just copy the design without thinking twice (so I will need to support that in many other websites).

And you, what do you think about it? How are you designing your A/B tests not to fall in this trap?

47 comments

[ 2.9 ms ] story [ 112 ms ] thread
[deleted]
(comment deleted)
jacquesm
It is the first time for me that I think that changes to the google homepage are really a step in the wrong direction. In the past they seem to have had it together on this one, if there were changes they were subtle and did not seem to get in the way of getting the job done.

Now they've gone overboard on the 'b(l)ing' factor, presumably to compete with other search engines that look 'spiffy'.

I'd rather have 10 text links (or even 30) per page without adornment than all the stuffy they've been adding lately.

Duckduckgo could easily capitalize on that, give us the very early google interface with better results.

That would make me switch, their scroll-down-to-get-more trick already is very elegant.

Loic
Thank you for reminding me about Duckduckgo, I always dismissed it because of the javascript requirement, but trying again today I found it pretty clean. I will need to commit myself to a one week long test to better assess how good it is.
jacquesm
While you're testing, try gigablast.com as well, they're pretty impressive.
epi0Bauqu
I'd appreciate you emailing me your feedback when you're done testing it.
[dead] jjvens
While we're at it, the reason I don't use Duckduckgo is that I can't see the search text when using the Gnome 'dark' theme. It seems they have used CSS to specify the foreground color (black) but not the background color (on my system: also black) of the text box.
drKarl
I guess it's probably easy to create a Stylish user style to revert changes.
pbhjpbhj
You should provide some image links when commenting on visual appearance especially WRT A/B testing as other users will get a different experience by definition.

Google homepage is like http://imgur.com/sc5h6.jpg for me so I'm guessing you mean the logged in personal home?

Loic
Your screenshot is for the home page before moving the mouse, of course you see nearly nothing, so it is consistent. Anyway, here is a commented screenshot of the search results page (not all the comments fit in, especially the font size on the left for the tools, so I did not include them). This is especially this one annoying me because I access google through the search box of my browser.

http://xhtml.net/documents/images/new-google-page.png

isleyaardvark
I don't like inconsistencies either, but I don't mind them as long as they're consistent within a conceptual section.

The arrows on the sidebar, for example, cause a different behavior when clicked than the arrows up top. The pipes between the links on the right are between links relating to your personal settings as opposed to the links on the left, which are filters for the search results (with the exception of Mail, which is inconsistently placed).

Overall I like the new look but I can see how it may not be the most cohesive design.

pbhjpbhj
That's a Google SERP not the homepage.

Yes the visual design sucks. I'm not there for the visual design and it doesn't suck enough to interrupt the utility for me.

msy
Looks like Douglas Bowman was right and its only got worse since he bailed. http://stopdesign.com/archive/2009/03/20/goodbye-google.html
Loic
Thank you! It was exactly this post I was looking for when writing this note and was unable to find it.
[deleted]
(comment deleted)
jessriedel
Bowman is more describing why he doesn't like working there; he really doesn't make a claim that Google is making poor design choices from a business standpoint. He criticizes the use of data (aka, you know, evidence) as a crutch without giving a single example, theoretical or otherwise, of how reliance on data rather than the gut instinct of a good designer leads to a bad outcome.
lzimm
I think its the same difference between investing in company lead by a former-consultant/bean-counter/"professional"-CEO and a real founder.

data is important, but it only gets you so far: table stakes. the thing that differentiates us is the ability to use intuition.

don't forget that your "gut" has been developed over millions of years of evolution to keep you alive, and its done a pretty good job. playing with numbers? couple thousand?

jessriedel
Intuition/experience is important when there is some aspect of reality that is hard to measure. Insofar as the data tracks your target, intuition is unnecessary. That's why I would want an example in which data was failing.

For instance, a fast food chain might take some data and find that they can decrease the quality of their ingredients without immediately loosing customers because people have habits and don't really notice right away (and because consumers will chalk up a few bad experiences to variance at a place they know well). But, over time, the chain develops a reputation for lower-quality food, and customers slowly leave.

The problem with this example is that the data can easily measure long-term effects, especially the subtle damage to reputation. This is a case where intuition may be needed to override data.

But where's the similar example with Google?

yesimahuman
This fascinates me because I see google as something that basically every single person I know uses. They don't need any more convincing. Who does google need to convince anymore? I understand there's competition, but they are already good enough in the looks department to become a basic web-use instinct. Why change?
symptic
Seems to me they do it to continue innovating. A company who stops innovating is one who then can only grow through conservation and focusing on efficiency. Now whether or not they are spending time innovating in the right areas is a debate to be had, but my personal opinion is the new layout sucks. For Google, it feels bloated.
_delirium
This is commonly talked about in the design field as well (since at least the 1960s, probably earlier). On the one hand, there's a very industrial-psychology-derived HCI tradition, based on user-testing everything with statistical significance and so on, that tends to do relatively small changes, e.g. assuming that most of the design will stay as it is, how about widget A versus widget B? Against them, there's more of a design tradition, derived from architecture, graphic design, art, and other areas, that looks at more holistic sorts of issues, and tends to think that doing 1000 A/B tests does not add up to "design", and isn't likely to produce good outcomes either. HCI hasn't really resolved the issue over several decades, and it flares up now and then (my impression is that the main HCI conferences/journals are ~80% the industrial-psych sort, but the 20% dissenters have persistently stuck around for decades).
msy
There is a fundamental misunderstanding of design at play here though. We're talking about a culture that doesn't understand that when it comes to design, 1+1 should equal 3 or more. You cannot separate out individual elements and A/B them without looking at the bigger picture. The net result is this - a kind of unsettling, uncomfortable inconsistency. The element level granularity of design hierarchy & planning is obvious. I love agile and I'm a fan of fast iteration and lots of user testing but a strong, consistent design requires top-down discipline and bold decisions, you cannot increment your way there.
_delirium
Yeah, I agree with that. It's not even particularly surprising from a scientific perspective: any scientist knows that if you measure A, B, C, and D, you don't necessarily know anything about union(A, B) or union(A, B, D), unless you also measure pairwise and higher-order interactions. At some point, instead of doing more A/B tests than there are atoms in the universe (since this explodes exponentially), it might be worth thinking about the problem using human intelligence.
swah
I kept adjusting the contrast of my monitor until I reailized the logo was changed.
joubert
Experimenting on customers.
mahmud
I have been stuck in an A/B hell myself, there is a fat sidebar to my left that I can't get rid of. Googling for a fix turned up a page saying it was an experiment and will be over soon.

For the last week I have been using Bing.

oneplusone
Delete your cookie for google.com. It drove me nuts too.
mahmud
I have some of the most aggressive security settings. And to be sure, I went back and deleted all cookies again. No dice.

Well, this will give me an excuse to sample other search engines. I was a heavy Altavista user until I found this new search engine out of Stanford ..

paraschopra
May be they were sampling based on user (you must be logged on?).

While doing such large A/B tests, they should definitely give an option to opt-out. Or at least a prominent feedback button to tell them how bad is their new design. It is as if Google will A/B test a new design and no-one will notice it?

samd
Perhaps another way of saying this is that Google's A/B testing has merely taken them to a local maximum, but they need a strong, coherent design discipline to reach the global maximum.
endlessvoid94
I mean....

these are such tiny gripes. Some arrows are different because they aren't of the same priority..people will click "More" more often than they'll click 'more search tools', so that arrow is smaller. makes sense to me.

How exactly is the search button inconsistent with the homepage? You think it would look better BELOW the textfield on the SERP?

I don't see any mixed case anywhere.

Not trying to be a troll or bust balls or anything, I just find your criticisms a bit nitpicky. The overall design of the site is incredibly simple and easy to use.

Loic
What really annoys me is that you have no space between the input field on the SERP (first time I discover this acronym) and the search button. On the homepage you have some space.

Also the advanced search link is now below, not anymore right from the input, it kinds of make this new SERP cluttered. Note that when logged in, it is even worse, I have another link for the safe search.

Also you will note that you have 4 or 5 different size of fonts.

I agree with you, roughly it is working (it what the data prove anyway, else they would have not done it) but nevertheless for me it feels wrong.

http://xhtml.net/documents/images/logged-in-google-serp.png

nate
What exactly are they optimizing or testing for here? Just number of searches, or clicks on that feature. Have they eliminated that people might be clicking on all these odd ball changes because its novel? I know I've expanded some menus and clicked on stuff and hovered over shit I wouldn't normally just because these changes were all shiny and I wanted to see what they do.

But in the end, the only change I like is the logo. Everything just looks a little odd. But my only real gripe is the location of the search results. Is it just me, or have the serps been left justified or were they much more in the center of the screen before? Every friggin machine is getting wide monitors, so to make stuff go left these days and just give me all this whitespace in the center and the right, makes me a little nuts.

arihant
Talking about the new sidebar, I really think it doesn't blend very well with the rest of the page. Compare this to the sidebar at www.clusty.com which feels so much "at home".
alayne
Well, I'm losing my religion. I already have a Greasemonkey script to stop the fading top row. Look at the results page for "cat toys" in Bing and Google. Maybe Google should just redirect to Bing.

I get that they're trying to satisfy a diverse user base and drive UI changes from testing, but it sure came out ugly.

Khaki
The results seem pretty much the same to me.
[deleted]
(comment deleted)
thorax
Yeah, what struck me was that if style was your differentiator for using Google versus Bing, you just lost your motivation to stick with Google.

Some examples:

Bing's style: http://www.bing.com/search?q=mortgage

New Google style: http://www.google.com/search?q=mortgage

Compare to the old google look: http://www.darrinward.com/img/articles/screenshot-google-sea...

puredemo
Excellent post. I had no idea how similar the Bing and Google result pages are now. What is up with that? It couldn't have been unintentional, could it?
kyro
Here's my problem with the design: With the old one, my eyes could act like magnets, being quickly attracted to either the left or right side, sliding up and down either side when viewing search results / ads. Now I have to do this balancing act, putting in quite a bit of effort to keep my eyes on the center column. Whereas I previously just let the magnetic forces direct my attention with the old design, I now have to resist the pull from both sides to keep center focused, making it extremely uncomfortable. Part of me thinks this new design probably yielded more ad revenue because now people can more easily succumb to the pull from the right. It's a lot less polarized than before.
adam_feldman
This is exactly how I feel. I have Adblock, but there is still a very strong pull to the left column that's distracting as I'm searching.
Pheter
I find it much more comfortable to use Google now that the results are nearer the middle of the page. I feel my eyes moving slightly to the left, but I recon that this is just an artifact of using the old Google for many years. Only time will tell, but I recon that this will quickly become a non-issue as I get used to the new design.

I still find the new design to be dissapointing, but i'm glad that the results are now pushed closer to the middle of the page; it's where the content is on most websites.

piramida
Just a thought - it may feel uncomfortable because people never accept changes, we all need time for brain patterns to rebuild. My first impression was - get the old page back. Second impression was - it now exposes some functionality I usually had to go to advanced for. I'd wait a month before jumping to any conclusions.

Yes there are inconsistencies but nothing 3 lines of code won't fix.

Keyframe
Google is know for many things good, design not being one of them. Sure, minimalism was their thing and it was good, but once they went in for design it showed that they have no clue. Usability yes/maybe, but aesthetics definitely no.
mr_justin
Nobody else in the world has anything even close too as many visitors as google, so it's really hard to apply normal logic to this testing they are doing. My feeling is that every single tiny and seemingly insignificant change will have an impact on their bottom line. If it affects 0.1% of their users, that's still a whole ton of people. I see nothing wrong with this multivariate a/b testing.
udfalkso
Who uses the Google home page anymore?