“There are additional complications in certain languages, where the comparison is context sensitive and depends on more than just single characters compared directly against one another,
[…]
Numbers. A customization may be desired to allow sorting numbers in numeric order. If strings including numbers are merely sorted alphabetically, the string “A-10” comes before the string “A-2”, which is often not desired. This behavior can be customized, but it is complicated by ambiguities in recognizing numbers within strings (because they may be formatted according to different language conventions). Once each number is recognized, it can be preprocessed to convert it into a format that allows for correct numeric sorting, such as a textual version of the IEEE numeric format.”*
I think those file browsers made the right choice, even given that they don’t (as in this example) always do the right thing.
I almost always want the version-sorting that's being presented in this article, rather than an "alphabetical" sort. But on the other hand, it absolutely seems like a valid bug that this is presented as an "alphabetical" sort, rather than something like "alphabetic/numeric" or similar. In other words, a problem of labeling rather than one of sorting.
If I understand the article, the author wants magic :)
I take it to mean they want the system to know file_9.txt is less then file_10.txt.
I never saw that happen in any OS, so I do not know what he is referring to. Maybe whatever that old system was, it sorted by create time as opposed to file name.
So, the author can try and create "aisort" that will look at all file names and add leading zeros to the file numeric portion, sort, then remove the zeros added. That will probably as slow as s***t and use gobs pf memory, depending on the number of files.
Digits and any other characters that are not A-Z and a-z should not get sorted. That's the true result of doing what you asked and not what you meant. Pedantic, but that's why we are here.
"I created the Alphanum Algorithm to solve this problem. The Alphanum Algorithm sorts strings containing a mix of letters and numbers. Given strings of mixed characters and numbers, it sorts the numbers in value order, while sorting the non-numbers in ASCII order. The end result is a natural sorting order."
When you get a bunch of files (let's say 1000+) without leading zeroes, this is a blessing. But I get the author's frustration, the expected behavior is not there, instead, he gets magical sorting that is wrong for his use case. I'm not sure what the ux should be, and maybe the algorithm here could be smarter, but it's a trade-off.
I get it, but if all these major operating systems are handling this same ambiguous [0] situation in the same way, perhaps one needs to reevaluate their mental model or expectations.
Am I out of touch? No, it's the operating systems who are wrong
I thought this was pretty well known. E.g. the macOS Foundation library even exposes NSString.localizedStandardCompare() [1] which implements the sorting algorithm used by Finder, and should be used by any well-behaved macOS application. Windows uses StrCompareLogical [2].
Not this keyboard not this chair, but the problem is with idiots between keyboards and chairs.
The author is not the ID10T it’s the other general users.
The author is intelligent enough to recognize that this is not alphabetical sort, but the term that they are looking for to describe the sort that they see in dolphin windows, google etc. is *lexical* sort, not alphabetical.
The engineering problem is ID10Tic not technical. How do you educate an illiterate public on what the difference between alphabetical and lexical sort is in practice?
You can’t, so you engineer around it and call lexical sort alphabetical.
I agree with Microsoft/Google/KDE's order. The author's situation is extremely rare, and the situation where someone wants "10" to be before "9" is far more common. Moreover, desktops don't label this sorting "alphabetical" (E: and it would really be "lexicographic"*), they label it "by name" (an informal criteria), so technically they're not lying.
> I miss the time when computers did what you told them to, instead of trying to read your mind.
You may be looking at that time through rose-tinted glasses. I don't like when computers lie to me either, but "mind-reading" is really helpful in ways we take for granted, like autosave. Desktops can have an option to sort files truly alphabetically, but the more common case should always be the default; that's the definition of "intuitive".
"mind-reading" is a really an unfortunate term though. Every algorithm is a strict and consistent set of rules that tries to serve the needs of its users. No magic is ever involved.
It is just that some users have conflicting needs and some sets of rules are more complex than others. So I think what this really is about is 'computer reading', the needs of some users to be able to predict with ease what the computer is going to do. Some people would rather be able to predict the computer doing something that they actually don't really need, and then make up for its shortcomings, than have something they feel they cannot predict and control, but is actually closer to what they want.
This is a bit like the term magic. Any sufficiently complex algorithm may indistinguishable from mind-reading, but it's still an algorithm. Mind-reading, like magic, depends on us being able to understand or not, which is highly subjective. But both are misleading terms.
Most of the time, as a regular user, I agree with having smarter ordering. And smarter all features for what its worth. Except when it doesn't work because of some corner case. In which case the "smart feature" becomes a kind of a leaky abstraction - now as a user I have to figure out how the machine works, so that I can trick into doing what I need.
Give the user an option: have both "by name" lexicographic ordering, make it default by all means, but also provide a way to switch to an alphabetical order one for power users. Same applies to other features.
It is disappointing that apps and even some Linux Desktops today take the flexibility away from users, in the name of usability. By all means, I like and benefit from all the smart features, and I want them and will keep the on by default, but leave me an option to do the simpler, dumber and more predictable things too, for the case when I need to fallback to it.
How is that right, when file explorer picks an arbitrary character in the middle(!) of the filename and sorts by it? Say, I have a file987name.txt and list5.txt, so sorting by name ascending a file explorer would for whatever reason decide to sort by fifth character, so that list5 would lower than file987name, because 5 is lower than 9, via some twisted logic. How is that normal in any way?
Thankfully I'm using Total Commander and FastStone as a image organizer, neither of which have this bug in the sorting.
Well, lots of interfaces don’t say “alphabetical” anymore, they say “name” or some variant, and then they can define it however they want, regardless or because of the frustration it causes users but not some other users which will now be inverted for long term-frustration averaged user experience.
> But 1 is smaller than 9, so file-10.txt should be first in alphabetical order. Everyone understands that, and soon people learn to put enough leading zeros if they want their files to stay sorted the way they like.
No. Not “everyone understands that”. Natural sort happens in real life and everyone understands that. Only those who understand ASCII — not the average user of graphical file managers — will deduce the reason for your definition of “alphabetical order”.
> Now that I know what the issue is, I can solve it by renaming the files with a consistent scheme.
> Well, apparently all these operating systems have decided that no, users are too dumb and they cannot possibly understand what alphabetical order means. So when you ask them to sort your files alphabetically, they don’t. Instead, they decide that if some piece of the file name is a number, the real numerical value must be used.
Well, no. You don't actually ask them to sort in alphabetical order. You ask them to sort "by name", and that is up to their interpretation. And they choose the interpretation that (per their reasoning, and possibly some actual data) seems most likely to correspond to what the user wants.
Maybe future versions of those OSes will add a rule that says that if any of the number groups have leading zeros then it reverts back to actual alphabetic order. Or maybe they'll give you configurable options. (Maybe some of them already do.)
> Of course, the user who named those files probably wants file-9.txt to come before file-10.txt. But 1 is smaller than 9, so file-10.txt should be first in alphabetical order. Everyone understands that, and soon people learn to put enough leading zeros if they want their files to stay sorted the way they like. Well, apparently all these operating systems have decided that no, users are too dumb and they cannot possibly understand what alphabetical order means. So when you ask them to sort your files alphabetically, they don’t. Instead, they decide that if some piece of the file name is a number, the real numerical value must be used.
I think there are many things wrong with your assessment of the situation.
First, where does it say in these file managers that they're sorting by alphabetical order? I see that you've specified that you want the files sorted by name, but I don't see that you've specified you want them sorted by name alphabetically. And what does "alphabetical sort" even mean when you're sorting characters which are not letters? What you mean is probably "lexicographical sort".
Second, you admit yourself that users probably want natural sort. Why would you expect these products to do the thing which they know users usually don't want by default? That just seems like bad design to me. They know users usually want natural sort, and you know users usually want natural sort, so why would you expect the default behaviour to be a lexicographical sort?
Third, just like how you've learned to work around the lack of natural sort in poorly designed products of years past by adding leading zeroes, you can just add trailing zeroes to get the lexicographical ordering that you want. Why do you seem to be implying that the latter is more user-hostile than the former? It doesn't make sense to me. A decision had to be made about what sort to use and they picked the one that most people want. Isn't that what we should be expecting in a product that caters to its users?
I see in other comments you've suggested that there should be a separate option for choosing between lexicographical sort and natural sort. But in the past, when lexicographical sort was the only option, why weren't you complaining about it being user-hostile to only have one option then? Why is it only when the default is something you're personally not used to that it warrants complaint? And where do we stop, do we have separate controls for every single sortable string field to determine whether it should be sorted lexicographically or naturally? Or just the name field? Don't you think that is going to lead to interface bloat?
To answer the question in the article, I’m pretty sure Windows Explorer (and probably File Manager before that) has sorted filenames this way for at least 30 years.
The author wants the "worse" sort, one based on ASCII/Unicode codepoints, without any intelligence for numbers that 99% of GUI users want.
For their purposes, they've assumed something about the implementation, to the point that a convenience feature is actually a misfeature for them. But the author here is probably a developer, or close to one, so they do not represent the needs of most people using computers.
Understanding the target audience for your product results in very different design decisions. Better is better might be great for products, but worse is better is probably better for systems that need to grow and evolve.
I think the algorithm is probably incorrect. A number starting with 0 should be treated lexically not numerically. Otherwise you have a situation where img_1_01.jpg and img_01_1.jpg does not have a complete ordering.
I got used to naming files/folders with leading zeros when I want them to be sorted alphabetically (for example payslips/invoices, etc).
But I'm a tech guy, I know what does "alphabetically" mean in the tech world. And it probably is not what common folks mean when they think "alphabetically" outside the tech world.
Edit: in fact, if I recall correctly, the proper term for this kind of sort (the one OP wants) is alphanumeric sort.
I've encountered a tangential problem to this with package versioning on Linux distros. Thankfully it was not too hard to write an algorithm to compare versions (thanks AI!).
103 comments
[ 3.1 ms ] story [ 73.3 ms ] thread“There are additional complications in certain languages, where the comparison is context sensitive and depends on more than just single characters compared directly against one another,
[…]
Numbers. A customization may be desired to allow sorting numbers in numeric order. If strings including numbers are merely sorted alphabetically, the string “A-10” comes before the string “A-2”, which is often not desired. This behavior can be customized, but it is complicated by ambiguities in recognizing numbers within strings (because they may be formatted according to different language conventions). Once each number is recognized, it can be preprocessed to convert it into a format that allows for correct numeric sorting, such as a textual version of the IEEE numeric format.”*
I think those file browsers made the right choice, even given that they don’t (as in this example) always do the right thing.
I take it to mean they want the system to know file_9.txt is less then file_10.txt.
I never saw that happen in any OS, so I do not know what he is referring to. Maybe whatever that old system was, it sorted by create time as opposed to file name.
So, the author can try and create "aisort" that will look at all file names and add leading zeros to the file numeric portion, sort, then remove the zeros added. That will probably as slow as s***t and use gobs pf memory, depending on the number of files.
https://web.archive.org/web/20210207124255/http://www.daveko...
Am I out of touch? No, it's the operating systems who are wrong
0 - numbers are not part of the alphabet.
1. Sync all equipments to the same clock.
2. Sort by Date Taken, if unavailable, sort by Date Created.
[1] https://developer.apple.com/documentation/foundation/nsstrin...:)
[2] https://learn.microsoft.com/en-us/windows/win32/api/shlwapi/...
The Finder sorts these as:
Whereas `ls -l` gives meNot this keyboard not this chair, but the problem is with idiots between keyboards and chairs.
The author is not the ID10T it’s the other general users.
The author is intelligent enough to recognize that this is not alphabetical sort, but the term that they are looking for to describe the sort that they see in dolphin windows, google etc. is *lexical* sort, not alphabetical.
The engineering problem is ID10Tic not technical. How do you educate an illiterate public on what the difference between alphabetical and lexical sort is in practice?
You can’t, so you engineer around it and call lexical sort alphabetical.
> I miss the time when computers did what you told them to, instead of trying to read your mind.
You may be looking at that time through rose-tinted glasses. I don't like when computers lie to me either, but "mind-reading" is really helpful in ways we take for granted, like autosave. Desktops can have an option to sort files truly alphabetically, but the more common case should always be the default; that's the definition of "intuitive".
* https://news.ycombinator.com/item?id=45404022#45405279
It is just that some users have conflicting needs and some sets of rules are more complex than others. So I think what this really is about is 'computer reading', the needs of some users to be able to predict with ease what the computer is going to do. Some people would rather be able to predict the computer doing something that they actually don't really need, and then make up for its shortcomings, than have something they feel they cannot predict and control, but is actually closer to what they want.
This is a bit like the term magic. Any sufficiently complex algorithm may indistinguishable from mind-reading, but it's still an algorithm. Mind-reading, like magic, depends on us being able to understand or not, which is highly subjective. But both are misleading terms.
Give the user an option: have both "by name" lexicographic ordering, make it default by all means, but also provide a way to switch to an alphabetical order one for power users. Same applies to other features.
It is disappointing that apps and even some Linux Desktops today take the flexibility away from users, in the name of usability. By all means, I like and benefit from all the smart features, and I want them and will keep the on by default, but leave me an option to do the simpler, dumber and more predictable things too, for the case when I need to fallback to it.
Thankfully I'm using Total Commander and FastStone as a image organizer, neither of which have this bug in the sorting.
No. Not “everyone understands that”. Natural sort happens in real life and everyone understands that. Only those who understand ASCII — not the average user of graphical file managers — will deduce the reason for your definition of “alphabetical order”.
> Now that I know what the issue is, I can solve it by renaming the files with a consistent scheme.
Intensely ironic given the previous suggestion.
Well, no. You don't actually ask them to sort in alphabetical order. You ask them to sort "by name", and that is up to their interpretation. And they choose the interpretation that (per their reasoning, and possibly some actual data) seems most likely to correspond to what the user wants.
Maybe future versions of those OSes will add a rule that says that if any of the number groups have leading zeros then it reverts back to actual alphabetic order. Or maybe they'll give you configurable options. (Maybe some of them already do.)
I think there are many things wrong with your assessment of the situation.
First, where does it say in these file managers that they're sorting by alphabetical order? I see that you've specified that you want the files sorted by name, but I don't see that you've specified you want them sorted by name alphabetically. And what does "alphabetical sort" even mean when you're sorting characters which are not letters? What you mean is probably "lexicographical sort".
Second, you admit yourself that users probably want natural sort. Why would you expect these products to do the thing which they know users usually don't want by default? That just seems like bad design to me. They know users usually want natural sort, and you know users usually want natural sort, so why would you expect the default behaviour to be a lexicographical sort?
Third, just like how you've learned to work around the lack of natural sort in poorly designed products of years past by adding leading zeroes, you can just add trailing zeroes to get the lexicographical ordering that you want. Why do you seem to be implying that the latter is more user-hostile than the former? It doesn't make sense to me. A decision had to be made about what sort to use and they picked the one that most people want. Isn't that what we should be expecting in a product that caters to its users?
I see in other comments you've suggested that there should be a separate option for choosing between lexicographical sort and natural sort. But in the past, when lexicographical sort was the only option, why weren't you complaining about it being user-hostile to only have one option then? Why is it only when the default is something you're personally not used to that it warrants complaint? And where do we stop, do we have separate controls for every single sortable string field to determine whether it should be sorted lexicographically or naturally? Or just the name field? Don't you think that is going to lead to interface bloat?
https://news.ycombinator.com/item?id=27916370
The author wants the "worse" sort, one based on ASCII/Unicode codepoints, without any intelligence for numbers that 99% of GUI users want.
For their purposes, they've assumed something about the implementation, to the point that a convenience feature is actually a misfeature for them. But the author here is probably a developer, or close to one, so they do not represent the needs of most people using computers.
Understanding the target audience for your product results in very different design decisions. Better is better might be great for products, but worse is better is probably better for systems that need to grow and evolve.
But I'm a tech guy, I know what does "alphabetically" mean in the tech world. And it probably is not what common folks mean when they think "alphabetically" outside the tech world.
Edit: in fact, if I recall correctly, the proper term for this kind of sort (the one OP wants) is alphanumeric sort.
[1] https://en.m.wikipedia.org/wiki/Afferbeck_Lauder
Usually preferable, except when not. Just like distinguishing between upper- and lowercase letters, and other misery.
[0] https://en.wikipedia.org/wiki/Natural_sort_order
When would it be preferable to distinguish between capital and lowercase letters?
Wiktionary does it religiously, and it always makes the entries worse. Want to know what something means in German? Well, that's on a separate page.
Do you want to look something up while using your phone? Don't be stupid; use a desktop that won't autocapitalize the first letter you type in.