The expected behavior is a default natural number sort.
What other default was chosen? String comparison for some value of Strings, coerced from the integers in some way.
If you have to research a bunch of implicit behavior to understand how sort works for numbers. I believe that's unintuitive by definition (simplest case is more complex than necessary). Fixable with a required flag or specialized functions (eg PHP).
> If `compareFunction` is not supplied, all non-undefined array elements are sorted by converting them to strings and comparing strings in UTF-16 code units order. For example, "banana" comes before "cherry". In a numeric sort, 9 comes before 80, but because numbers are converted to strings, "80" comes before "9" in the Unicode order.
Javascript has some weird behaviors, but in this case Arrays in Javascript can contain multiple types. How would you sort an array of `['cat', 80.3, '80']`? I think I would expect that to end up as `['80', 80.3, 'cat']`, not `[80.3, '80', 'cat'].
Or to make things more complicated, `['cat', 80.3, { foo: 'bar' }, true]`. Do we want to try and intuit whether that object gets converted to a string or a value? Do we base that intuition on the prototype chain, or what?
I think it makes sense to cast every object/type in the array to a shared format so everything can be compared based on the same criteria.
---
Have simple rules that are universally applied:
- All objects are converted to strings
- Strings are sorted alphabetically
vs something weird like:
- We try to intuit what type you want based on several common type conversions and possibly the prototype
- We have a set of built-in rules about which primitive types (number, boolean, string, Symbol) comes before and after other primitive types.
We don't know that the items are clearly numbers, arrays in Javascript are untyped.
Your proposal is that Javascript's default sort algorithm should change behaviors based on whether it can do an extra O(n) loop over the entire array and check whether `typeof x === 'number'` evaluates to true for each index? I don't think that's a good idea.
This is one of the reasons I am so excited about WASM. I am playing around with the Rust framework yew (https://yew.rs/docs/en/intro/) to replace a lot of my front-end Javascript programming, and it is a joy to use.
If you love Javascript that is wonderful, but I am happy to not be forced to use it, when I would prefer other languages.
nobody thinks that this behavior is good. the problem is, this is how it was at the beginning, and javascript will not break backwards compatibility. so we have to live with it (or use a third-party sort-function, like "lodash.sortBy").
there are other similar problems in javascript too:
- another problem with ".sort()" is that it both mutates the input-array and returns the mutated input-array. so you write code like "const arr2 = arr1.sort()", and the code works, but if you don't know the spec, you will not know that "arr1" was modified and basically "arr1 === arr2".
I know this seems weird at first glance, but JS arrays are not typed. To all the people saying this is obviously wrong, I'm curious what your intuitive default strategy is to sort the following array:
The big thing everyone is always complaining about with Javascript is that it tries to guess too much. Type conversions with equality operators, type conversions with +/- operators, lots of Javascript doing unexpected conversions and trying to be helpful at the wrong times.
So sure, you could have a set of random rules based on prototype chains to figure out whether you were going to call `valueOf` or `toString` for any given object, and you could come up with a bunch of rules for whether or not primitive types like Symbols come before numbers.
Or, you could just say, "we're going to convert every value to its string representation and sort alphabetically." At which point, as long as you understand how string conversion works, it becomes very obvious and easy to understand how the array above will get sorted.
The second option is easier to learn and easier to reason about. There's weird stuff in Javascript, but if you're going to have a default (and there's good reason to say that you shouldn't, and that `sort` without a parameter should just throw an error), I think this is the right choice for them to have made about how the default should work.
Lots of intuitions about what the developer meant seem simple when you're only looking at the simple case. But it is usually better to have one set of rules that are easy to describe and that are universally applied. More of Javascript should work this way, if we're going to have a default sorting algorithm, then it's good that the default sorting algorithm mostly does the same stuff every time no matter what you pass it.
This is not the kind of thing that makes me dislike Javascript, it's the other stuff -- the inconsistencies and magic behaviors -- that give me more pause about the language. But what Javascript is doing here is good.
2) Throws an error if two of those types are non-comparable
I think that's a much better approach than the implicit casting that happens, frankly, all throughout JavaScript. I like JavaScript - more than Python, overall - but this is one of those things where I don't think we need to be defending the unfortunate decisions that were made in the past.
> Throws an error if two of those types are non-comparable
The problem is that practically everything in Javascript is comparable. This doesn't really solve the sorting conundrum, it just rephrases it. What's the intuitive way to handle:
If your assertion is that all of those should just throw errors and we should stop doing type inference entirely, then that's kind of reasonable, I guess, but that's a much more drastic change to how Javascript works. If the assertion is that thinking through Javascript operator conversions is easier than thinking through string conversions, I just disagree with that.
Javascript's operator conversions are more complicated and have more weird rules and edge cases. They are (somewhat justifiably) one of the biggest things people complain about in the language. The last thing I want is to have to explain to a new programmer why `5 < '10'`, and `5 > '3'`, but also `'5' > '10'`. Given the way that Javascript is set up at its core, I think using comparison operators for a default sort method would be a disaster.
Using `<` as it's implemented today would mean that even a simple array like `[5, '10', '3']` would sort non-deterministically depending on what the sorting algorithm was and what order the indices were compared with each other. It's unusable unless you're going to change how Javascript operators work.
> The problem is that practically everything in Javascript is comparable
> If your assertion is that all of those should just throw errors and we should stop doing type inference entirely, then that's kind of reasonable, I guess
The thing is that the above isn't actually type inference, it's type casting. There's a difference. And in JS, implicit casts happen not just in "reasonable" cases like int -> float, but between every possible type:
"[object Object]" == {} // true
This, to my mind, is JavaScript's greatest remaining flaw in 2020. It's the whole reason the triple-equals (===) operator was added, but that only solves one common case.
So my basic argument is: in totally unreasonable implicit-casting scenarios (which is most of them), it's much better for a language to err than to make a "best effort" guess at casting the values to be the same type.
All of that said: JavaScript is the way it is, and unlike lots of other things that have been papered-over with new features over the years, this issue is so foundational that I don't think it can ever be fixed outside of just learning to avoid the places where it shows up.
> this issue is so foundational that I don't think it can ever be fixed outside of just learning to avoid the places where it shows up.
Agreed. And the reason why the default array sort behavior is good is because it avoids one of the places where that shows up.
The alternatives were:
- don't have a default method at all (arguably the correct choice)
- fundamentally change how Javascript works, breaking the web in the process
- turn the default sort method into an arcane, unpredictable monster
By comparison, "turn everything into a string" is pretty reasonable behavior. Given what Javascript is, string conversion isn't a gaffe or thoughtless design decision, it is the correct behavior for a default Javascript sort function to have.
Nobody is suggesting changing JavaScript's behavior in-place. But there is a more reasonable behavior that could be used for a hypothetical "sort2()" method: just don't cast the values eagerly.
You can still do the cast if needed when they're different types (once the < operator is applied). Whatever, that's the norm for JavaScript. But in this case, the cast to a string happens before it even knows it needs to, which is what makes the OP's case so egregious. In this case all the values are the same type - numbers - and they still get needlessly cast to strings. IMO, waiting until the < to do any casting is not only more reasonable overall, but is actually more inline with expected JS behaviors.
If you don't do eager casting, you get inconsistent sorting behavior depending on the algorithm and unsorted list order.
Under the current algorithm,
randomize_array(['3', 5, 10, '11']).sort()
will always give you
[10, '11', '3', 5]
Under your proposal, where our casts are dependent on what two elements we're currently comparing, you might get
[10, '11', '3', 5]
or
['3', 5, 10, '11']
Try it yourself:
['11', 10, '3', 5].sort((a, b) => a < b ? -1 : 1);
[10, 5, '3', '11'].sort((a, b) => a < b ? -1 : 1);
The only way that type-specific sorting algorithms work is if we check the entire array before we start sorting, which has performance implications. It can't be on-the-fly, we either have to commit to type-checking the entire array, or commit to casting the entire array. I think casting is more consistent, I don't like the idea of an algorithm changing underneath me because of a type error. But in either case we can't change our sorting behavior mid-sort.
This is part of the reason why recommendations on how to sort an array of integers is to use a sort method like
(a, b) => a - b
instead of the comparison operator -- because most of the time subtraction will cast everything to a number across the entire array, including the strings. For most people, this will be fine. To head off what would logically be the next thing to suggest, we also can't rely on `a - b` for a default sorting algorithm for `sort2`, because of
['11', 10, '3', 5, {}].sort((a, b) => a - b);
[{}, 10, 5, '3', '11'].sort((a, b) => a - b);
Most of the examples you provide are meaningless. What is the intuitive way of comparing a number and a function? There isn't one, the question itself is meaningless. Saying "hey look we found a way to come up with an answer by turning everything into a string" isn't right either, since it means you come up with the wrong answer in plenty of situations—like in the original tweet.
I'd rather have a library that gives you the correct answer in important common cases, at the expense of not being able to answer questions that don't have an answer. The solution JS went with was to get the important cases wrong, so that it could handle cases that don't have a meaning or use.
> I'd rather have a library that gives you the correct answer in important common cases, at the expense of not being able to answer questions that don't have an answer.
The problem is that optimizing for the common cases and ignoring the difficult or meaningless cases is exactly how you end up with 20 different rules for how something should work that all handle the situation kind of well, but interact with each other to create horrible, crazy bugs whenever anything unexpected happens.
Look, Javascript type conversions are a disaster, I'm not going to pretend otherwise. But the reason they are a disaster is specifically because the language was so worried about making everything intuitive that it didn't stop to think about what would happen when the half-dozen 'helpful' type conversions interacted with each other in novel situations.
It's better to have one or two rules that work everywhere, even if they occasionally surprise a new programmer. Beginners will only be beginners for a small, finite amount of time, but they will need to deal with your language quirks and emergent weirdness for the rest of their lives.
If more of the core Javascript language embraced that philosophy, it would be a better language to program with.
It's the "don't worry about the edge cases, just make the obvious case right" thinking that got us class syntax that doesn't actually build classes, and strings parsed into numbers for `>` but not `+`, and basically the entirety of `==`. Because all of those situations had slightly different answers about what was the "intuitive" thing to do, and nobody thought any farther than the isolated, immediately obvious use-cases for those features.
> The problem is that practically everything in Javascript is comparable.
No, the problem is that JavaScript has not been replaced by something better before now. The problem is that when JavaScript was written, it was never, ever intended for the things it is being used to do today, and backwards compatibility requirements prevent the very necessary, very long overdue destruction of JavaScript.
The problem is not comparability, the problem is JS itself.
> No, the problem is that JavaScript has not been replaced by something better before now.
I suspect replacing Javascript was an out-of-scope task for the people trying to design a `sort` function.
> and backwards compatibility requirements prevent
WASM will work just fine without breaking existing websites. People can hate on Javascript all they want, but "don't break the web" is a good policy that we should be encouraging in most cases. If anything, there's been a kind of harmful trend from browsers like Chrome in the opposite direction, and I think it's bad for the web to make it into a less stable platform.
> JavaScript will forever be a very successful toy because of things like this. Types are important, and the tweet begins to expose why.
The "everything should be strongly typed" argument is not one that I'm necessarily unsympathetic to, but I also just don't feel like it's relevant here.
Javascript isn't strongly typed. What did you expect the spec authors to do, break the entire web just so that they could have a better `sort` method? They built a decent algorithm given the language and paradigms that they were building on.
Soon everyone who hates Javascript will be able to use Rust/C/C# targetting WASM, and hopefully these pointless "it's a toy language" debates will calm down once that option goes more mainstream.
47 comments
[ 5.4 ms ] story [ 36.9 ms ] threadNo. It's not obvious.
Python -
PHP - Rust - The expected behavior is a default natural number sort.What other default was chosen? String comparison for some value of Strings, coerced from the integers in some way.
If you have to research a bunch of implicit behavior to understand how sort works for numbers. I believe that's unintuitive by definition (simplest case is more complex than necessary). Fixable with a required flag or specialized functions (eg PHP).
JavaScript is weakly typed, so it's never appropriate to assume such a thing without applying a type conversion. Sad, but true.
> "Straightforward"
Again, this is JavaScript.
[6, -2, 2, -7].sort();
The result comes back with this unexpected set:
[-2, -7, 2, 6]
> Default sort converts each element to string and orders lexicographically.
https://chromium.googlesource.com/v8/v8/+/4.3.18/test/mjsuni...
I'm definitely not saying "You're using it wrong".
> If `compareFunction` is not supplied, all non-undefined array elements are sorted by converting them to strings and comparing strings in UTF-16 code units order. For example, "banana" comes before "cherry". In a numeric sort, 9 comes before 80, but because numbers are converted to strings, "80" comes before "9" in the Unicode order.
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe...
(Relevant xckd: https://xkcd.com/293/)
maybe we could have a `sort2()` method that works differently.
People are fast to complain..
Javascript has some weird behaviors, but in this case Arrays in Javascript can contain multiple types. How would you sort an array of `['cat', 80.3, '80']`? I think I would expect that to end up as `['80', 80.3, 'cat']`, not `[80.3, '80', 'cat'].
Or to make things more complicated, `['cat', 80.3, { foo: 'bar' }, true]`. Do we want to try and intuit whether that object gets converted to a string or a value? Do we base that intuition on the prototype chain, or what?
I think it makes sense to cast every object/type in the array to a shared format so everything can be compared based on the same criteria.
---
Have simple rules that are universally applied:
- All objects are converted to strings
- Strings are sorted alphabetically
vs something weird like:
- We try to intuit what type you want based on several common type conversions and possibly the prototype
- We have a set of built-in rules about which primitive types (number, boolean, string, Symbol) comes before and after other primitive types.
What do YOU expect when you ask someone to sort items that are clearly numbers?
I don't know about you, but I do not expect what actually happened.
Your proposal is that Javascript's default sort algorithm should change behaviors based on whether it can do an extra O(n) loop over the entire array and check whether `typeof x === 'number'` evaluates to true for each index? I don't think that's a good idea.
If you love Javascript that is wonderful, but I am happy to not be forced to use it, when I would prefer other languages.
https://www.destroyallsoftware.com/talks/wat
there are other similar problems in javascript too:
- "typeof null === 'object'". this is very wrong, but again,this is how it was implemented at the beginning... you know the rest ( https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe... )
- another problem with ".sort()" is that it both mutates the input-array and returns the mutated input-array. so you write code like "const arr2 = arr1.sort()", and the code works, but if you don't know the spec, you will not know that "arr1" was modified and basically "arr1 === arr2".
So sure, you could have a set of random rules based on prototype chains to figure out whether you were going to call `valueOf` or `toString` for any given object, and you could come up with a bunch of rules for whether or not primitive types like Symbols come before numbers.
Or, you could just say, "we're going to convert every value to its string representation and sort alphabetically." At which point, as long as you understand how string conversion works, it becomes very obvious and easy to understand how the array above will get sorted.
The second option is easier to learn and easier to reason about. There's weird stuff in Javascript, but if you're going to have a default (and there's good reason to say that you shouldn't, and that `sort` without a parameter should just throw an error), I think this is the right choice for them to have made about how the default should work.
Lots of intuitions about what the developer meant seem simple when you're only looking at the simple case. But it is usually better to have one set of rules that are easy to describe and that are universally applied. More of Javascript should work this way, if we're going to have a default sorting algorithm, then it's good that the default sorting algorithm mostly does the same stuff every time no matter what you pass it.
This is not the kind of thing that makes me dislike Javascript, it's the other stuff -- the inconsistencies and magic behaviors -- that give me more pause about the language. But what Javascript is doing here is good.
1) Uses < for whatever types are present
2) Throws an error if two of those types are non-comparable
I think that's a much better approach than the implicit casting that happens, frankly, all throughout JavaScript. I like JavaScript - more than Python, overall - but this is one of those things where I don't think we need to be defending the unfortunate decisions that were made in the past.
The problem is that practically everything in Javascript is comparable. This doesn't really solve the sorting conundrum, it just rephrases it. What's the intuitive way to handle:
If your assertion is that all of those should just throw errors and we should stop doing type inference entirely, then that's kind of reasonable, I guess, but that's a much more drastic change to how Javascript works. If the assertion is that thinking through Javascript operator conversions is easier than thinking through string conversions, I just disagree with that.Javascript's operator conversions are more complicated and have more weird rules and edge cases. They are (somewhat justifiably) one of the biggest things people complain about in the language. The last thing I want is to have to explain to a new programmer why `5 < '10'`, and `5 > '3'`, but also `'5' > '10'`. Given the way that Javascript is set up at its core, I think using comparison operators for a default sort method would be a disaster.
Using `<` as it's implemented today would mean that even a simple array like `[5, '10', '3']` would sort non-deterministically depending on what the sorting algorithm was and what order the indices were compared with each other. It's unusable unless you're going to change how Javascript operators work.
> If your assertion is that all of those should just throw errors and we should stop doing type inference entirely, then that's kind of reasonable, I guess
The thing is that the above isn't actually type inference, it's type casting. There's a difference. And in JS, implicit casts happen not just in "reasonable" cases like int -> float, but between every possible type:
This, to my mind, is JavaScript's greatest remaining flaw in 2020. It's the whole reason the triple-equals (===) operator was added, but that only solves one common case.So my basic argument is: in totally unreasonable implicit-casting scenarios (which is most of them), it's much better for a language to err than to make a "best effort" guess at casting the values to be the same type.
All of that said: JavaScript is the way it is, and unlike lots of other things that have been papered-over with new features over the years, this issue is so foundational that I don't think it can ever be fixed outside of just learning to avoid the places where it shows up.
Agreed. And the reason why the default array sort behavior is good is because it avoids one of the places where that shows up.
The alternatives were:
- don't have a default method at all (arguably the correct choice)
- fundamentally change how Javascript works, breaking the web in the process
- turn the default sort method into an arcane, unpredictable monster
By comparison, "turn everything into a string" is pretty reasonable behavior. Given what Javascript is, string conversion isn't a gaffe or thoughtless design decision, it is the correct behavior for a default Javascript sort function to have.
You can still do the cast if needed when they're different types (once the < operator is applied). Whatever, that's the norm for JavaScript. But in this case, the cast to a string happens before it even knows it needs to, which is what makes the OP's case so egregious. In this case all the values are the same type - numbers - and they still get needlessly cast to strings. IMO, waiting until the < to do any casting is not only more reasonable overall, but is actually more inline with expected JS behaviors.
Under the current algorithm,
will always give you Under your proposal, where our casts are dependent on what two elements we're currently comparing, you might get or Try it yourself: The only way that type-specific sorting algorithms work is if we check the entire array before we start sorting, which has performance implications. It can't be on-the-fly, we either have to commit to type-checking the entire array, or commit to casting the entire array. I think casting is more consistent, I don't like the idea of an algorithm changing underneath me because of a type error. But in either case we can't change our sorting behavior mid-sort.This is part of the reason why recommendations on how to sort an array of integers is to use a sort method like
instead of the comparison operator -- because most of the time subtraction will cast everything to a number across the entire array, including the strings. For most people, this will be fine. To head off what would logically be the next thing to suggest, we also can't rely on `a - b` for a default sorting algorithm for `sort2`, because ofMost of the examples you provide are meaningless. What is the intuitive way of comparing a number and a function? There isn't one, the question itself is meaningless. Saying "hey look we found a way to come up with an answer by turning everything into a string" isn't right either, since it means you come up with the wrong answer in plenty of situations—like in the original tweet.
I'd rather have a library that gives you the correct answer in important common cases, at the expense of not being able to answer questions that don't have an answer. The solution JS went with was to get the important cases wrong, so that it could handle cases that don't have a meaning or use.
The problem is that optimizing for the common cases and ignoring the difficult or meaningless cases is exactly how you end up with 20 different rules for how something should work that all handle the situation kind of well, but interact with each other to create horrible, crazy bugs whenever anything unexpected happens.
Look, Javascript type conversions are a disaster, I'm not going to pretend otherwise. But the reason they are a disaster is specifically because the language was so worried about making everything intuitive that it didn't stop to think about what would happen when the half-dozen 'helpful' type conversions interacted with each other in novel situations.
It's better to have one or two rules that work everywhere, even if they occasionally surprise a new programmer. Beginners will only be beginners for a small, finite amount of time, but they will need to deal with your language quirks and emergent weirdness for the rest of their lives.
If more of the core Javascript language embraced that philosophy, it would be a better language to program with.
It's the "don't worry about the edge cases, just make the obvious case right" thinking that got us class syntax that doesn't actually build classes, and strings parsed into numbers for `>` but not `+`, and basically the entirety of `==`. Because all of those situations had slightly different answers about what was the "intuitive" thing to do, and nobody thought any farther than the isolated, immediately obvious use-cases for those features.
No, the problem is that JavaScript has not been replaced by something better before now. The problem is that when JavaScript was written, it was never, ever intended for the things it is being used to do today, and backwards compatibility requirements prevent the very necessary, very long overdue destruction of JavaScript.
The problem is not comparability, the problem is JS itself.
I suspect replacing Javascript was an out-of-scope task for the people trying to design a `sort` function.
> and backwards compatibility requirements prevent
WASM will work just fine without breaking existing websites. People can hate on Javascript all they want, but "don't break the web" is a good policy that we should be encouraging in most cases. If anything, there's been a kind of harmful trend from browsers like Chrome in the opposite direction, and I think it's bad for the web to make it into a less stable platform.
It doesn't make sense to sort an array containing different types at all so why is this even allowed at all?
JavaScript will forever be a very successful toy because of things like this. Types are important, and the tweet begins to expose why.
The "everything should be strongly typed" argument is not one that I'm necessarily unsympathetic to, but I also just don't feel like it's relevant here.
Javascript isn't strongly typed. What did you expect the spec authors to do, break the entire web just so that they could have a better `sort` method? They built a decent algorithm given the language and paradigms that they were building on.
Soon everyone who hates Javascript will be able to use Rust/C/C# targetting WASM, and hopefully these pointless "it's a toy language" debates will calm down once that option goes more mainstream.
Don't sweat the small stuff.