HN Explorer

Ask HN: HN web stack?

31 points by kaiser_pelagic ↗ HN
I'm curious about the HN web stack. I know they use arc as the icing but I have no idea what is underneath.

server: Apache(mod_scheme) or plt(racket) web server are my two guesses db: MySQL, PostgreSQL - I read somewhere that they use MySQL

20 comments

[ 2.9 ms ] story [ 57.8 ms ] thread
Darmani
http://www.arclanguage.com/install

You can read the source code yourself. I don't remember the database, but it's running on an Arc server.

kaiser_pelagic
I have taken a look at the source, but wasn't sure if they were using all of it in production. If they are then quite impressive.
jackolas
Being proxied to Server:Apache/2.2.8 (Ubuntu) fyi

edit: I don't think the page itself is actually proxied it seems only the static content is.

ryan-allen
If the app is behind apache and writing static files for serving things up, and only hitting arc on operations like votes and cache misses then of course it's gonna scale! Especially if the app is read heavy :)

Is the HN source code actually open source itself?

epochwolf
I thought the database was flat files.
[deleted]
(comment deleted)
known
Yes, everything is a file in UNIX
omnix
or OP could be trying to hack hackernews
jrockway
And what a crisis that would be. All of our publicly-available comments would become available publicly!
silentbicycle
No database. Just flat files cached in memory on demand, IIRC.

I think about that when people worry about scalability. HN seems to do okay.

kaiser_pelagic
using flat files just seems crazy to me, but HN seems to be doing it well.
cloudhead
Why? A database is just an optimized flat file with a query engine.
rlmw
Because a database is an optimized flat file with a query engine. Its commonly the case that you wish to optimize your flat files' performance and query them. I believe that pg has mentioned a few times that he's 'notorious' for using flat files for this kind of thing. I'd love to see the code for Viaweb and the design decisions in there.
j_baker
HN has scaled, but not because pg hasn't worried about scaling (although to be fair, it doesn't seem to have worried much):

"I already have problems enough with that. Remember, the original motivation for HN was to test a new programming language, and moreover one that's focused on experimenting with language design, not performance. Every time the site gets slow, I fortify myself by recalling McIlroy and Bentley's famous quote

The key to performance is elegance, not battalions of special cases.

and look for the bottleneck I can remove with least code. So far I've been able to keep up, in the sense that performance has remained consistently mediocre despite 14x growth. I don't know what I'll do next, but I'll probably think of something."

http://paulgraham.com/hackernews.html

silentbicycle
Sure. Profiling and incremental tuning has been sufficient so far. It will probably be hard to scale indefinitely, but HN already handles quite a bit more traffic than many sites.
jjoe
It's running the Arc web server, which has its own implementation of the HTTP protocol. I don't think there's anything behind or in front of it. Probably not even Memcached. Think of the app serving HN as a framework. Here's how a "URL" entry would be implemented in Arc:

(defop hi req (prn "Hi there"))

http://news.ycombinator.com/whoami will return your login and IP. It's probably implemented in this fashion:

(defop whoami req (prn (req 'ip)) (prn (req 'id)))

Regards

Joe