HN Explorer

Hacker News would scale better with some caching headers

14 points by sayrer ↗ HN
sayrer$ curl -I http://news.ycombinator.com/news

HTTP/1.0 200 OK

Content-Type: text/html; charset=utf-8

Connection: close

6 comments

[ 4.0 ms ] story [ 27.4 ms ] thread
ROFISH
It is being done for the images. I assume no caching for the front page since it's assumed the front page changes enough to not warrant it.

ryan$ curl -I http://ycombinator.com/images/y18.gif

HTTP/1.1 200 OK

Date: Tue, 24 Mar 2009 05:04:58 GMT

Server: Apache/2.2.10

Last-Modified: Sun, 18 Mar 2007 17:03:07 GMT

ETag: "7fa02b-64-42bf6725588c0"

Accept-Ranges: bytes

Content-Length: 100

Content-Type: image/gif

jwilliams
More likely: -- the content for the front page is served by Arc, whereas the static media is served from a separate Apache instance.

Arc's current HTTP server libary (src.arc) only supports HTTP/1.0.

sayrer
Well, HTTP doesn't really have versions other than as a shorthand for header defaults. For example, Connection: close handles implementations that assume HTTP 1.1, iirc.

An Expires header would help a lot for arc-generated pages served in response to requests that aren't associated with a logged in user.

fhars
According to the Yslow developers, even thse replies could be made far mor cache friendly by using the correct headers, especially by removing the ETags-header that forces a conditional GET on every acccess and replacing it with an Expires-header in the far future that allows to reuse the cached version without server interaction (assuming the images are not meant to be highly dynamic, then ETags would make sense).

So from a caching perspective, HN has got it exactly backwards: it doesn't use ETags on the dynamic pages, where it might help, but uses it on the static content, where it might be detrimental. But I wouldn't bet on the gains you'll get from fixing this to be measurable.

whughes
There're three images on the whole site. Is it really making a difference?
Confusion
A question that must be asked before investing any time in adding to an existing application: would it solve an existing problem? The Right Thing isn't always necessary and often not worth the time.