Do you mind sharing some details? This happens to a lot of sites so it would be interesting to know what your initial configuration was and what you scaled up to.
I would love to see an analysis from someone who gets HN'd/Slashdotted/Reddit'd often traffic stats. I know that the "frontpage" of reddit would garner you hundreds of thousands of hits.
There are heaps of blog posts on HN, digg, reddit, slashdot etc from overly excited dudes who can't wait to show us what a one day spike then decline back to nothing looks like in Google Analytics.
What exactly are the 10 instances? Did you add VPS's or just more child processes to handle connections?
If you haven't already, it's good practice to calculate how much resources one instance/thread/process of your server takes up, and then allocate as much as your server allows. (of course take into consideration shared memory/copy-on-write when calculating limits)
I don't see 139 when I navigate the mincraftforum.net directly, but when you have 8 ad units on a page, plus site analytics and a sharing toolbar or two the 37 that I see is about what i'd expect.
One thing that could be happening to drive the number up in some cases is identity synchronization between advertising providers. Vendors like BlueKai share profile data about you (sites visited etc) by calling out to the domains of other partners, who redirect back with an ID appended (or vice versa). This ID match then enables out-of-band profile sync.
Best spot to do this is these sites who go for ad overkill.
It's worth noting that the user ID sync only needs to happen once per bidder in the real-time auction that determines which ads get served - once the bidder's user ID is linked to the auction provider's user ID, no more cookies need to get set.
A typical user who landed on this page as part of their normal everyday browsing wouldn't have nearly so many cookies written - because this site is being tested in isolation, it's a bit misleading.
This is probably why the parent here only saw 37 cookies, vs. the tester's 139.
The software acts like a first-time visitor with no set cookies and empty cache. I agree on the fact that once you're logged in, made a bid or whatever, the number of cookies written on each page load is substantially lower... :)
This is the real cause of most cookie deletion. (EDIT: Many) browsers are configured by default for ~1000 cookies, and all these ad networks and data providers setting their own cookies quickly overrun that. It doesn't help that google Analytics sets separate (first party) cookies for each website you visit.
That 1000 cookie limit becomes a sort of FIFO, and that's why we always seem to get logged out at weird times for no reason.
One case could be that an ad agency allows customers to direct a subdomain to their servers (eg. ads.domain.com). A visitor doesn't know if the request is 'close' by or to some shady ad agency...
It's not a website owned or operated by notch, it's completely independent (or at least, it was when it was created). I think it's mad because it probably is degrading user experience, just not in exceedingly noticeable ways (i.e. sending 139 cookies for every request takes time, so each page load will take more time).
Why would anyone keep 3rd party cookies enabled? I have found only 2 or 3 sites that needed to be whitelisted. Everything else works just fine without them.
What sites, out of curiosity? Obviously all the like-like (likesque) buttons would no longer function, but I don't care about those. I was wondering what valid use case there is for 3rd party cookies that actually benefits the user and not the service provider.
In case anyone is interested, in Chrome (Mac at least), it's under Settings > Show advanced settings > Content Settings (under Privacy) and then the checkbox "Block third-party cookies and site data". http://www.evernote.com/shard/s8/sh/eb3a23ef-8902-4ae1-87d8-...
So I guess this is one reason why people leave that enabled: not quite obvious.
They also include scripts from (at least) 9 different external domains, similar the minecraftwiki and they monitor page activity. Every few seconds a ping packet about browser information is sent to some random advertisement corp.
Nothing special if you just want to make the most money possible with a community, that besides hosting, you have absolutely nothing to do with :(
36 comments
[ 4.2 ms ] story [ 79.0 ms ] threadUnfortunately I seem to have lost the bandwidth stats, but the load is the cool one. And my poor VPS didn't actually crash through all that.
Happy cookiechecking!
If you haven't already, it's good practice to calculate how much resources one instance/thread/process of your server takes up, and then allocate as much as your server allows. (of course take into consideration shared memory/copy-on-write when calculating limits)
One thing that could be happening to drive the number up in some cases is identity synchronization between advertising providers. Vendors like BlueKai share profile data about you (sites visited etc) by calling out to the domains of other partners, who redirect back with an ID appended (or vice versa). This ID match then enables out-of-band profile sync.
Best spot to do this is these sites who go for ad overkill.
A typical user who landed on this page as part of their normal everyday browsing wouldn't have nearly so many cookies written - because this site is being tested in isolation, it's a bit misleading.
This is probably why the parent here only saw 37 cookies, vs. the tester's 139.
That 1000 cookie limit becomes a sort of FIFO, and that's why we always seem to get logged out at weird times for no reason.
[1] http://src.chromium.org/chrome/trunk/src/net/cookies/cookie_... search for kMaxCookies
if lots of cookies is a viable technique, the browser vendors will increase the cookie limit.
Then again, 640k of ram is enough for anyone.
[1] http://peter.upfold.org.uk/blog/2010/05/12/centralised-comme...
So I guess this is one reason why people leave that enabled: not quite obvious.