Ask HN: Why are developers so stingy with “rows per page”?

As a non-designer developer, I'm going to say this is dictated by UX designers and has no technical merit. My preference trends towards "put about 10k rows into the browser and let the user use ctrl+f to find what they're looking for" which is probably also wrong for all sorts of reasons.

Only thing worse than pagination is infinite scrolling.

Because there's no point in loading 500 rows from the database if 90% (made-up stat) of people don't read past the first 10.

- Query time: Developers are very squeamish about how long SQL queries take

- Payload size: 1000 rows makes the payload huge which adds even more latency

- Frontend performance: More modest machines will struggle to render very large dynamic tables of rich content.

(I am not a real lawyer) but I think it's just little more complex to program. CPU time issues don't matter much, we can design around that.

If we pull, say, 1000 rows in the foreground, you have to keep in mind when designing UI that you have to wait for data. E.g. you have to design some sort of indication that data is not yet 100% loaded if user tries to search something in page.

Another point is that (maybe, I have no experience in UI design) that users favor fast loading page, even if it only has 10 rows..

There is a psychological aspect too. Show people 10 results and they'll actually look at them. Show people 100 results and they'll not only quickly scroll through them but also skip the top results, which is counterproductive.

The big sites have definitely AB tested changing the results per page and have stuck with 10 for some user metric driven reason.

I'm positive this is usually due to convention, not a conscious design decision. In other words: websites give 20/50/100 row pagination options because other websites give 20/50/100 row pagination options.

There is no meaningful implementation difference between providing 20/50/100 options and 20/50/100/500 options, with 20 as the default in either case.

I would be very surprised if this decision is even discussed for more than 1 minute on most projects, let alone if any user research is done. But I'm happy to be proved wrong!

Some comments suggest this is as a result of A/B testing, which may be part of the story today, but as someone who built websites that had to deal with returning large resultsets from time immemorial (i.e. pre-dot com bubble), it absolutely started because Netscape would simply blow up rendering tables larger than N rows (where N ain’t very large) and because partial rendering didn’t work very well with table based layout. The former meant you could crash a viewer’s browser while the latter meant anything more than say 20 rows would take eons to load over a dialup modem.

This all changed with the advent of infinite scrolling enabled via the whole ajax revolution, but this was (is still? I haven’t written front end code in a decade or so) difficult to get right.

In sum, this happened because of technical difficulties on the front end side (browser limitations or code complexity), and probably just stuck as convention. Perhaps some nice analyst then A/B tested the arrangement and found that it was happily optimal.

From a UX, perspective, Nielsen Group is a good reference: https://www.nngroup.com/articles/infinite-scrolling/

For your question specifically, the following explain it well I think:

..locating a previously found item on an extremely long page is inefficient, especially if that item is placed many scrolling segments down. It’s much easier for people to remember that the item is on page 3 than it is to gauge where the item is positioned on an extremely long page.

Yes, search facilities on pages are usually not great, it's easier to show the raw data and use browser search...

It's interesting to see this. I feel the same way, hate card-based layouts and generally want more information density (i.e., HN or old Reddit) as opposed to what's often done now.

I intentionally put all of my Elixir screencasts on a single page without much whitespace on Alchemist Camp (https://alchemist.camp/episodes). I can't think of a single employer who would have been cool with that design choice, but I frequently get emails and comments from people thanking me for it.

CPU time is cheap, human attention is not.

I would not assume most users want a lot of choices. Perhaps choosing from a list of 5-10 options is much less intimidating than choosing from 100. If someone clicks next page, then maybe the calculus changes.

One is technical.

20,000 rows on search can kill the server, depending.

20,000 rows displayed can kill the users PC.

One is the fact programmers are generally stupid. They cargo cult. I'll often give backend users unlimited rows if they chose. This is the programmers job, not the product owner or the UI designer who are even dumber, programmers need to stop passing the buck for their shonky work.

I do wonder if it adds or detracts from $$$ sold. I'm pretty sure online markets benefit from inefficiencies. There are lots of other options that would make search better they don't do. In part people wandering might make $.

For forums it'll be about user page views for ads.

A chrome add on that blows it out to 10,000+ rows for forums, commerce etc would be handy.

If there is a shortage of cpu time, then please use pagination instead of dynamically loaded infinite scrolling.

I hate how so many websites manages to break the back button.

I like the idea of geared pagination https://github.com/basecamp/geared_pagination which is another gem out of Basecamp.

Their philosophy is to show less initially but as you start paging through additional pages it'll show more results per page until you hit a maximum amount. For example you could get 25 results on page 1, 50 on page 2, 100 on page 3, etc..

It's a happy medium. Keep your initial result minimal and focused but if a user wants more then keep giving them more in an efficient way (less clicking).

If there is advertising revenue involved, then the pagination is all about hits. Each page load is an opportunity to serve more ads.

On the web it can definitely become a performance constraint, unfortunately. Frameworks can fold under even this number of rows if you're, for example, re-rendering your entire VDOM tree on every keystroke in an input field. This part can be optimized around fairly easily if you know how, but then the next barrier is number of DOM nodes (specifically, the layout calculations that may have to be done on them when even something seemingly unrelated on the page changes). Depending on the complexity of your rows and what else may be going on on the page, you're probably still confined to the triple-digits if you want smooth performance.

Source: I spend a couple years wrestling constantly with these sorts of problems in a React-based toolset we were developing.

One strategy I've used before to improve the user experience and help people miss Ctrl+F less is to load the full data set into memory, even if only a small slice of it is actually rendered at a time, and then do all paging/searching/sorting/filtering client-side. JavaScript has no issue handling six digits of items this way, and it keeps things really snappy.

In my experience, it's the query time. Maybe it should be faster, but practically, with all the tech debt that goes into these things, it's not. I've actually experimented with different sizes for a social media app (meaning it queries total reacts, shares, etc) and 15 was close to optimal.

Latency is not an issue, and neither is displaying complex information. You can pull and display 400 dummy items no problem.

Loading time for the first "page" is extra valuable - you'd want that in 200 ms or so ideally. So one trick is load 3 items, then 30 or so.

Also you have to look at actual cost. Perhaps loading an extra 15 queries on the home screen costs $0.0004, but when you have 1m daily active home page uses, that's an extra $400 per day. In unoptimized pieces of code, the cost could well be 20x higher.

If you have a very high average user value like Jira, that's fine. But for say, a free manga site or something like imdb, you want to shave off costs wherever possible.

A good search ui should allow you to specify number of hits, imo. But 1000 would present technical problems. I'm most experienced with React and it would be difficult to render that many rows of anything in a useful way.

Additionally, depending on how your backend is designed and your choice of ORM, sometimes requests can be slow. In those cases it probably makes sense to design a UI that encourages users to rely on search and filter controls vs giving them a large list.

1000 products could mean 1000 images to show and 1000 templates to render and that may kill your browser (definitely true on a phone), and significant slowdown even on a desktop/laptop. The solution is simply to load more data as you scroll. This should be fast enough for you not to even notice it.

Because Tables with many rows are SLOW.

I have one with 6K rows, 300ms to insert into page on paper (console.time(1); console.timeEnd(1)), but in reality browser freezes for 3 seconds (1.8s style, 900ms layout, 300ms update tree). Freezing goes away with position: absolute, but it still takes 3 seconds to show up after .appendChild. I tried replacing Table with Flex divs, even worse speed.

    table { table-layout: fixed; }

It's mainly out of respect for the user's bandwidth. For example, if they're on a 2GB phone plan, then 500 records could drain a significant % their monthly allotment.

Depends what you're displaying, js frameworks can struggle to show 1000's of rows smoothly, especially if a row is displaying computed data.

So, a lot of the time, developers will just limit the page size, in order to guarantee performance on majority of devices...

My bank does that. Want to print your monthly transactions? There you go. Oh no, it's just the first 10 rows with infinite scroll. Scroll, again and again and again, just so you can print to pdf.

The worst i ve seen is in Bluehost's domain management pages. An unsorted list of domains with infinite scroll where you have to scroll, scroll and scroll hoping that your domain will be in the next block that pops up (because, of course they keep all your long abandoned domains in the list). And they do it again in their DNS Zone file editor. It will only show you the first 10 or so lines of each section, so naturally you think lines are missing and try to re-enter them. The comparison with the plain old CPanel interface is mind boggling. I have complained, they don't listen

Progress!

I wonder how i can start a campaign to legally ban infinite scrolling