Ask HN: What are some rigorous ways to review and test usability of web apps?

I think the most most rigorous test is deploying in front of paying customers. Usability is not tidy and theoretical. It is practical and messy. The core of usability is functional usefulness not design aesthetics. At the scale of Google, material design may have an absolute difference of a million users. At the scale of the average app, it is probably close to zero difference. Apple's iOS skeumorphism wasn't driving iPhone sales, people used it because the iPhone was useful. Emacs is still going strong after nearly forty years. People use Vim.

Good luck.

I have a background in UX/HCI but havn't worked with it for ages, so no good examples but just some general thoughts.

I think you have some good ones on your list, but heres a few you may consider to add to it:

a) You criteria should take into account who the end user is, because that will define a lot of whats good or bad design/solutions in relation to your points 1-7. This is subdivided into the users skill with technology in general and the users domain expertise level. Is the end user a novice or expert with tech? Is the user (or rather the primary use-case(s)) for domain experts of domain novices? I think tech-ability is addressed fine by your 7 bullets (at least here on top of my head). The last one about domain expertise is the most interesting one to consider imho.

Here is an example: When we typically say that a user interface has to be user friendly or have good usability or whatever we call it, we picture a GUI with a visual nice design, well organised information, not too much not too little, it is easy to do stuff and so on. But what about the linux sysadmin - is a good interface for him a consistent theme, layout and coloured buttons etc? No, he just wants a terminal interface and great usability to him is if the darn thing is scriptable and supports unix piping input/output (or whatever is sexy to sysadmins).

In genereal you can say that if the user is a domain expert and the use-cases for the application is specialist work, then you want a user interface that is more complex and requires more time to learn -- and that is okay, because the expert user can invest 5-10 hours (or more) of learning the complex interface if it makes him e.g. 25% more productive for all of his tasks for the rest of the years he'll use the system. On the other hand, if the system is a landing page for a SaaS startup, you need to design something that is understandable and usable the target user in just about 20-30 seconds, otherwise he has clicked on. The last example here is a variant of the "domaine novice" situation, and note that it changes little wether the user is tech-novice or tech-guru.

NB: The same user will be a different kind depending on what part of his job he is carrying out (this also goes for non-work scenarios) - the engineer may be happy with a very complex application for his engineering tasks, but the system for handling expenses should be simple (b/c he knows nothing about accounting).

The crucial point is that good usability is very dependent on who the user is and what the use-cases are. And you need to build that into your criteria somehow.

b) I think you may be able to add a criteria on "expectations" - do the system (e.g. a button) do what the user expects it to do? This can be down to colouring and icons: The accept button should be green and not red, the floppydisc icon should save the document, not close it and so on. But there are also more complex things like what the system indicates will happen when the last page of a registration wizard is completed. This is basically Donald Norman kind of stuff I'm thinking about (affordances is the concept he is talking about. sadly became a bit overhyped amongst UX designers but the ideas and principles are great).

c) Last thing and also very important is to consider the context of the system (as in IRL context). If for example you are designing a system for doctors or first responders it will be a key part of "good usability" that you can be interrupted and turn away from the screen a lot, and quickly get back to the task in the system again. The obvious example is if you are automatically logged out after 1 minute of inactivity: If the natural workflow, that the system is part of, simply entails that the user will get interru...

As a rule of thumbs, it’s a good idea to conduct a heuristic evaluation as an initial assessment of usability, make changes as needed, and then conduct an actual usability test to ensure you hit on the points you need and your users can accomplish key tasks in your app...

An aside - I find it quite strange that HN has such a blind spot for user research. I’ve been in the UX/user research industry since the mid 2000s. It’s gone from being a niche thing (it was a struggle to find any employers in London who understood what user research was when I started out) - to being a standard practice in big tech companies. Apple, MS, Facebook, Spotify... They’re all at it.

It’s something you really need to know about if your work involves making any kind of decisions that impact your users.

- task oriented. Don't build a complex do-it-all UI, split it into tasks, one at a time

- use simple, clear language to support what you want to user to do

- have a flow to guide the user in the task, so that the user does not need to come back or jump all over in the page

- do the "mother test": show it to your old mom, if she understands then you are in the right direction, otherwise ask her what she gets and what she does not get from your app. Repeat until almost anyone can use your app

In computing, a web application or web app is a client–server computer program in which the client (including the user interface and client-side logic) runs in a web browser.

The last two words "web browser" are very important. Are you testing your app using a variety of browsers? Or, perhaps, you're requiring everyone to use IE 6? :) I'm surprised that such a fundamental issue was left unstated.

Once you test various browsers, be sure to go in and set the minimum font size to a reasonably big number. At least 18. Maybe larger. Why not 24? I don't care if the resulting appearance on screen looks like crap, as long as everything is functional. Far too many web pages and web apps obscure large amounts of their functionality once the font size is increased.

Ask me how I know that. :)

As a QA, I frequently have the following experience:

me: "This won't make sense to people."

dev: "Oh, but it has to be this way b/c {reasons}"

me: "People are going to get confused. They're going to use it wrong and make errors."

dev: "No, our users are smarter than that."

{6 months later}

dev: "We're redesigning our interface because the users found it too confusing."