For a couple minutes, Hacker News was returning 502 Bad Gateway error codes with the message that "We're having some trouble serving your request. Sorry!".
I noticed that as well, and also saw a similar "blackout" a few days ago that lasted longer. First time ever in years... so wondering if something has changed / is changing on their side.
Every week or so, HN will give me that error, but it's almost always back in a few minutes. I don't really think that much about it, sometimes we all have bad days.
My theory: The admins were trimming a whole bunch of submissions about Musk's takeover that all appeared at once, to try to force everything into one mega-thread submission. Everyone saw the news/tweet and rushed to HN to be the one who picks up all those precious HN karma points!
In any event, don't worry about it! There are other sites on the internet.
Scaling a tree structured view is difficult as hell. Elon musk thread shows just one single comment, and "view 557 more comments" button. I guess it's because of the number of comments written for the first comment. I would expect to see other threads in the news detail page, not only one, but probably it's some sort of optimization to handle enormous amount of comments.
I guess the twitter's way of handling huge threads is much more scalable and I think twitter's acquisition should had such a point.
> Scaling a tree structured view is difficult has hell.
Really? Why is that? It doesn't seem like that should be the case here. HN threads are only a few hundred comments, and the web page structure is very linear and formulaic (the threads do not form a tree in the DOM). I.e. it's a depth-first traversal visiting children by ranking, each comment producing a row in a table. For just a few hundred comments I would expect that to be very fast (but I don't know what data structure it's all stored in).
[1] It's not easy to determine how many root-level comments should be shown in the first page. You need to count sub-comments as well, and second level comments should have loaded as "expanded", it's also not easy to decide how many second level comments should be shown as expanded. The amount of them could be 10 and 100 or 1000.
So if you just say like "10" comments, the letters in just one comment could be worth of 100 comments. So you also need a "show more" button for each comment exceeds some character limit.
If you do a viewport optimization, you need to calculate the rendered size of a comment section and request only comments visible for the browser viewport.
I don't understand. We're still talking about trivial numbers. This is a tree with ~800 nodes or comments. There is no "dynamic viewport" optimization, HN is static HTML (and we love it for it!).
I was just thinking, what could be the problem in Elon Musk's thread. It shows just one comment. I didn't say there is a dynamic viewport optimization.
The issue is not in DOM structure.
I'm creator of "arguman.org", maybe you've heard about it (it's down now). I had the same issues there as well.
I'm also happy with HN's plain html interface.
Also, how would you render a page with 800 nodes with user content, it is still a question. I would like to hear and learn.
Btw, yes, Bulletin board style forums is a strait forward solution, there's always some ways to do that, but there's also ways to do it like in Twitter's way. That's the point I was talking about.
>Also, how would you render a page with 800 nodes with user content, it is still a question. I would like to hear and learn.
It would be trivial with any site using a relational database. The two basic models used are nested sets[0], which optimize rendering over insertion and deletion, versus the much simpler relational model (rows with id and parent id, sometimes root id), which is much easier to insert and delete from, but requires recursive queries to render an entire tree from any branch other than the root. The vast majority of threaded forums use the former model, many larger sites use the latter, and rendering large threads isn't an issue either way unless maybe you're at Reddit scale, which most forums including HN aren't.
Hacker News doesn't use a relational database, it uses flat text files (limiting it to file I/O speed for data not cached in RAM), and it also runs under a single thread with limited RAM. It likely wasn't designed to operate efficiently at the scale that it currently does, and the increasing size and velocity of threads is pushing it to its limit.
Optimizing the render part in insertion and deletion is also difficult. Some nodes might have user-level views, so some of them won't be visible to some other users. So you can't just label the nodes like this is "left" or "right" of that record.
My experience with a similar problem: Instead of putting the rendering part to the insertion and deletion (nested set model), I was using a simple nginx file based cache. This is a trade off and I chose the application layer to be "consistent" and "reliable", over "being fast". Keeping the data integrity is difficult when you have a nested set model and it's also expensive to doing that calculation during the insertion. Sometimes you need a message broker to handle such an expensive query, because it's not safe to do it within a request-response lifecycle, even if you do have a transactional database, there's always a barrier of "request time out".
It seems simple enough to just generate the conversation as one big page, then break it up. As you traverse the tree, save a list of indices of top-level comments and store the (recursive) sum of children sizes, then split appropriately. This might be slow with millions of nodes, but I don't understand how that process alone would be slow with hundreds.
What would be slow (and may be happening), is generating that dynamically for every page refresh on a busy story by every visitor (i.e. no caching).
Generating one big page is not the scalable way of doing that, I think, It will eventually reach to a point that it's no longer a web-page but a downloadable document with couple of megabytes.
I'm interested to brain-storm about how to model and implement "Breaking-up" part of your comment.
For the backend part, I think Splay-tree kind of a data structure will help to serve it faster.
I misspoke a little. I wouldn't generate the whole thing as a blob of text and then split that. I would traverse the tree depth-first making HTML blobs for new comments that that hasn't been done for already, generate a linear list of comment items that each know the size of their own subtree (set that when you traverse back up to the node), and make a list of the indices of the top-level comments as I go. Then you can go through the top-level comments, adding them (and their subtrees - everything up to the next top-level index in the list) one by one into your webpage and adding up the total size so far until you reach some threshold of "too big", at which point you finish off that page and start the next one.
I think caching would be important to make it fast, and you could cache the HTML of each comment or subtree. Actually, if I was making my own forum and not copying HN I would probably have the comment tree structure represented in the DOM tree, as that seems better for navigation and accessibility. All of this is totally theoretical of course! I haven't actually done it, so may be wrong.
> Scaling a tree structured view is difficult as hell.
It's not that hard with so few comments, it's just a matter of UX philosophy. At reddit we solved this with the "load more comments" AJAX request and "continue this thread". "Continue" only happens when a tree gets too deep, and then is just sets the current comment as the top of the tree and recalculates.
"load more" happens after the comment limit was reached, but since it's AJAX it doesn't really take away from the viewing experience (and I think in new reddit it just auto loads when you scroll to it).
The main issue for HN is that it's static so when a node gets a lot of children, it breaks a lot of their assumptions and makes for very large blobs of HTML to display the tree.
Honestly it's probably a worthwhile tradeoff -- for the most part the threads here don't get so deep that it's a problem, and in exchange we get a super snappy website most of the time.
I saw this for the first time last week and have since gotten this several times. Often opening the site in Incognito mode "fixes" it, but I would not expect a Bad Gateway error to returned in case of any user-specific throttling or anything so I put it down to a coincidence.
33 comments
[ 2.9 ms ] story [ 74.9 ms ] threadIn any event, don't worry about it! There are other sites on the internet.
Any suggestions? Sometimes I peek back at Slashdot. Otherwise Hacker News is my goto aggregator.
I guess the twitter's way of handling huge threads is much more scalable and I think twitter's acquisition should had such a point.
Really? Why is that? It doesn't seem like that should be the case here. HN threads are only a few hundred comments, and the web page structure is very linear and formulaic (the threads do not form a tree in the DOM). I.e. it's a depth-first traversal visiting children by ranking, each comment producing a row in a table. For just a few hundred comments I would expect that to be very fast (but I don't know what data structure it's all stored in).
So if you just say like "10" comments, the letters in just one comment could be worth of 100 comments. So you also need a "show more" button for each comment exceeds some character limit.
If you do a viewport optimization, you need to calculate the rendered size of a comment section and request only comments visible for the browser viewport.
That's why it's difficult.
* Elon Mush threads shows only one thread now. https://news.ycombinator.com/item?id=31025061
Edit: Looks like it is no longer an issue. It shows latest comments.
The issue is not in DOM structure.
I'm creator of "arguman.org", maybe you've heard about it (it's down now). I had the same issues there as well.
I'm also happy with HN's plain html interface.
Also, how would you render a page with 800 nodes with user content, it is still a question. I would like to hear and learn.
Btw, yes, Bulletin board style forums is a strait forward solution, there's always some ways to do that, but there's also ways to do it like in Twitter's way. That's the point I was talking about.
It would be trivial with any site using a relational database. The two basic models used are nested sets[0], which optimize rendering over insertion and deletion, versus the much simpler relational model (rows with id and parent id, sometimes root id), which is much easier to insert and delete from, but requires recursive queries to render an entire tree from any branch other than the root. The vast majority of threaded forums use the former model, many larger sites use the latter, and rendering large threads isn't an issue either way unless maybe you're at Reddit scale, which most forums including HN aren't.
Hacker News doesn't use a relational database, it uses flat text files (limiting it to file I/O speed for data not cached in RAM), and it also runs under a single thread with limited RAM. It likely wasn't designed to operate efficiently at the scale that it currently does, and the increasing size and velocity of threads is pushing it to its limit.
Edit: looks like the stone passed.
[0]https://en.wikipedia.org/wiki/Nested_set_model
Optimizing the render part in insertion and deletion is also difficult. Some nodes might have user-level views, so some of them won't be visible to some other users. So you can't just label the nodes like this is "left" or "right" of that record.
My experience with a similar problem: Instead of putting the rendering part to the insertion and deletion (nested set model), I was using a simple nginx file based cache. This is a trade off and I chose the application layer to be "consistent" and "reliable", over "being fast". Keeping the data integrity is difficult when you have a nested set model and it's also expensive to doing that calculation during the insertion. Sometimes you need a message broker to handle such an expensive query, because it's not safe to do it within a request-response lifecycle, even if you do have a transactional database, there's always a barrier of "request time out".
What would be slow (and may be happening), is generating that dynamically for every page refresh on a busy story by every visitor (i.e. no caching).
I'm interested to brain-storm about how to model and implement "Breaking-up" part of your comment.
For the backend part, I think Splay-tree kind of a data structure will help to serve it faster.
I think caching would be important to make it fast, and you could cache the HTML of each comment or subtree. Actually, if I was making my own forum and not copying HN I would probably have the comment tree structure represented in the DOM tree, as that seems better for navigation and accessibility. All of this is totally theoretical of course! I haven't actually done it, so may be wrong.
I agree. It's also good for screen readers, and you can also have keyboard shortcuts to navigate on tree.
It's not that hard with so few comments, it's just a matter of UX philosophy. At reddit we solved this with the "load more comments" AJAX request and "continue this thread". "Continue" only happens when a tree gets too deep, and then is just sets the current comment as the top of the tree and recalculates.
"load more" happens after the comment limit was reached, but since it's AJAX it doesn't really take away from the viewing experience (and I think in new reddit it just auto loads when you scroll to it).
Here's the reddit code if you want to see it: https://github.com/reddit-archive/reddit/blob/master/r2/r2/l...
The main issue for HN is that it's static so when a node gets a lot of children, it breaks a lot of their assumptions and makes for very large blobs of HTML to display the tree.
Honestly it's probably a worthwhile tradeoff -- for the most part the threads here don't get so deep that it's a problem, and in exchange we get a super snappy website most of the time.