HN Explorer

Ask HN: When did comments start being split across pages?

55 points by joshstrange ↗ HN
I searched for an announcement but I didn't find anything. On a recent post I was reading the comments [0] on I noticed a "more" link at the bottom to take me to the second page of comments, I had never seen that link before and wondered if it was new.

[0] https://news.ycombinator.com/item?id=13817557

47 comments

[ 4.1 ms ] story [ 220 ms ] thread
midgetjones
Maybe it's a workaround for the issue where HN hugs itself to death if comments get over a certain number.
noir_lord
If that's the case and given all the recent issues (including the somewhat hilarious "Can you all log out because S3 falling over is killing us EDIT: for clarity - Because of the level of auth'd user activity not because HN depended on S3" stuff) maybe it is time they actually considered fixing HN.
positr0n
I've seen the mods make the "please log out to save load on our server" announcement a few times on really busy threads.

My understanding is it wasn't because HN depends on S3 but because everyone was on HN at the same time talking about it!

Edit: my google/HN Search fu is failing to find a reference, but they can't easily horizontally scale HN because it is running on a single arc process. Comments/Submissions are stored in text files, and the "More" links are actually stored in memory in closures. When the closures get garbage collected you get the "Unknown or expired link." error.

noir_lord
Yeah I wasn't very clear about that I edited.

It was the volume of comments in a single thread.

[deleted]
(comment deleted)
foxylion
I'm really curious why the backend is in that state since so long. Today everything is scalable but HN performance is limited to a single process? And the complexity shouldn't be too high to migrate it to another framework/... when the current one does by design not support a multi process architecture. Do I miss something?
jafingi
The reason why that's happening is because when you're logged in, they can no longer server your response from the cache. So they have to render the page for _you_.
noir_lord
I know the reason why, it doesn't negate my point that given they are having so many issues recently (or possibly blind chance has made me see them having so many issues) it might be time to consider updating it.
drwl
I saw this too. So far I've encountered it on Who's Hiring threads.
ryandrake
It's extra painful on Who's Hiring threads because think--what do we all do? Full-page keyword search, right? Now you have to check if there's another page to search.
snowpanda
I prefer it the old way, but I understand why they might have changed it.
Raphmedia
If anything they should make it so that when you click "show more" it adds the next page to the current page instead of redirecting you to another page.
3131s
This 100%, or just put it all on one page. It's just text, so I really don't see why the change was necessary. I didn't notice it until yesterday until the CIA thread but it's really annoying.
jasonkostempski
That would require javascript, don't they try to avoid that?
chatmasta
No JS needed. You can do this with CSS using the target selector and display:none. But that would require loading all comments up front which is maybe what they're trying to avoid?
Raphmedia
Simply make it so that when JS isn't being used by the link acts normally.

You have two option with the web. Either you put your logic in the backend or in the frontend.

This addition is obliviously to reduce the load on the server. So either you add some logic on the frontend to make the user experience good or you do like they do right now and have a bad user experience.

dustinhayes
Not really, you could just load the same page with something like ?comments=all from the server.
jasonkostempski
That seems better actually, considering "more" doesn't really make sense when the things you've already gotten may have shifted in the mean time.
SippinLean
This is what reddit does, but that has its own problems: How do I ctrl+f the whole content? What if I refresh? What if I don't have JS? I haven't seen an elegant solution yet.
pmontra
I guess that the noscript behaviour should be to present the "More" link and let the user click it to load a new page. Exactly what we have now.

The JS in the page should attach an event to the link to append the new page at the bottom. Better: it should do it automatically when the scroll position is close enough to the end.

Bonus: the script should add a search box to search even in the content not yet in the page.

I googled if it's possible to intercept CTRL-F and handle it programmatically. It seems it is [1] but I don't know if I would do it: the user could be surprised to find a search box in a non standard position with a non standard behaviour. It reminds me of scrollbar hijacking.

[1] http://stackoverflow.com/questions/9159322/overriding-the-br...

Raphmedia
This opinion is never popular because for some reason developers seem to be dead afraid of asking their user's output.

What I would do is add a dialog the first time the user press CTRL+F. Asks if the user wants the default or the override. Save this in a cookie or in the database.

joshka
Alternatively, you could load all the threads when responding to ctrl+F for the first time on a page and just leave the browser default find UX alone.
SippinLean
"ctrl-f" was overly literal, I simply meant "a way to search the content of all the comments."

A simple (non-JS) "Search these comments" form would suffice, but I've never seen it implemented.

alva
It seems to have been implemented after the 2016 election threads. I remember dang being frustrated at the page loading times on threads with thousands of comments.
chippy
I'm sure some smart developer will write a browser extension to grab the next page and append it to the current one.
1123581321
I don't mind pagination if it's for mobile performance reasons, but I'd really like an option to disable it in my profile settings.
nilkn
I think this is a less than ideal workaround because it breaks (Cmd|Ctrl)+F search on large comment threads. I do this quite a bit just to see if some particular topic has already been discussed somewhere. Actually I think the current design is a little unintentionally insidious just because it doesn't make it clear that any comments are being hidden, so if not for seeing this I probably wouldn't have realized for quite a while that there was even a "More" link to click.
dmbtwy4dmbcmmnt
It's been this way for years. It's trivial to use the site search to pull up threads that have hundreds of comments. This doesn't prove whether those comments were spilling over back then, or whether you're seeing pagination being retroactively applied, but IIRC, this has been going on since at least as far back as 2013.
[deleted]
(comment deleted)
toephu2
I noticed it 2 days ago and no option to change it back. It really sucks when doing CTRL+F to search for stuff. Please give us an option to turn pagination off.
dang
At least as far back as 2013, but we only turn it on if the site is getting crushed. These days that happens when (a) there's a story on the front page with many hundreds of comments (say 750+) and (b) there's higher than usual traffic. That was the case yesterday with the big CIA/Wikileaks thread. We've turned it off now though.

One of you will now point out that it would be better not to have to do this (or, as someone gently put it downthread, "maybe it is time they actually considered fixing HN"). You're right!

noir_lord
That was me down thread, I wasn't intentionally been sharp, I love HN for the community of people/things that get posted and it's one of the few sites that actually works properly on my frequently flaky mobile internet so please don't go full Web 2.0 if you do fix things!.
dang
No worries—as far as I can tell we're in total agreement!
EGreg
Just out of curiosity, how do you crush a site with just posting maybe 100 text based comments on the top 20 items or whatever?

I mean what exactly is the load that causes this?

If you are executing some algorithm for each request - stop. Just run it every minute to rebuild a cache. HN discourages rapid fire replies anyway.

How hard is it to simply build an html fragment from a tree and cache it? You just see what nodes need other nodes to be appended.

OK, I can imagine MAYBE the most intensive thing is the votes and re-ordering of replies.

First of all I hope you are using this algorithm: https://medium.com/hacking-and-gonzo/how-reddit-ranking-algo...

Secondly, seriously once again you can batch the upvotes by thread and then process them during the recalculation phase. You only need to recalculate weight totals for the threads where there were upvotes - and this can be parallelized.

pvg
some algorithm for each request - stop. Just [...] How hard is it to [...] OK, I can imagine MAYBE

It can't be any harder than not coming off blithely condescending.

EGreg
Not condescending at all. Incredulous.

That's because the solution is straightforward, the people who built it are smart, had years to do it.

If I can build the basic scalable submission and comment system in one 8 hour day I am sure they can fix it in less than that.

Then you need to iterate of course and get the bugs out. But come on. I am incredulous that HN is buckling under a load of 20 stories with on average 100 comments each. Seriously?

pvg
If I can build the basic scalable submission and comment system in one 8 hour day

...

[unless this is deliberate parody in which case, you got me]

EGreg
of what
pvg
It feels like joke-explaining because it is but "I can implement this in a weekend" is one of the central self-unaware brogrammer cliches. You asked.
EGreg
Everything I say is self-aware and the result of reflection before posting.

And btw I said 8 hours, not a weekend.

Look up Poe's Law.

[deleted]
(comment deleted)
dang
The solution isn't straightforward because the problem isn't straightforward. It just looks like it is, which is by design.

The problem isn't straightforward because the software does a ton of things to try to keep the site good. We do a lot of manual moderation too, but anything we can write code for, we of course do.

iDemonix
It's comments like this that fuel my drive to earn HN points so I can get the downvote button.
tradersam
I'm with you. The annoying part is it is quite unobvious to find out how much karma you need for specific functions to open up.
krapp
>I mean what exactly is the load that causes this?

For one thing, other forums would use a database, whereas (from what I understand, but don't necessarily comprehend) Hacker News uses Lisp objects in memory, and flat files. We know how simple it might be to refactor in, say, PHP or Node but who knows what kind of eldritch Lisp weirdness this thing has going on under the hood?

joshstrange
dang, thank you so much for the explanation! It's more that I had hoped for, honestly I expect someone to just link to a recent announcement that my search/google-foo had missed so I'm glad to hear it was a while ago and that it's only a temporary thing as, like other's have mentioned, I use (Cmd|Ctrl)+F quite a bit and would hate to think I'm regularly missing out on discussion of certain topics because there is another page I didn't know existed. The whole thing makes complete sense and I'll just have to remember to scroll down to the bottom when viewing a post with a high number of comments.
krapp
It could also trim comments after a certain depth, like Reddit does.

HN may do that, but I've only seen it paginate by the number of subthreads. The bulk of new traffic in a thread is likely to be comments to existing threads. This would also make thread folding faster, because there would be a hard limit to the depth of any thread on a single page.