Linus is an incredibly egotistical ass. Yes, he's right about a lot of stuff. But when he's wrong, he cannot possibly even admit to that possibility.
Case in point, #6 there where he slams the HFS+ filesystem. This started an extremely long flamewar on the Git ML[1], between me and Linus, with everyone else piling in on the whole "Linus is always right" bandwagon. If you read the thread, you'll find that I was giving an explanation for why HFS+ stores filenames the way it does and why it actually makes sense in context, whereas Linus basically resorted to "it's crap because I say so". More specifically, he couldn't even admit that there were two approaches that had sensible arguments (filenames are human-readable strings, which HFS+ uses, vs filenames are sacred buckets of bits, which Linus believes). Instead, anyone who disagrees with him is obviously a moron.
Wrong about what? I clearly defined how there's two approaches to filenames. One is as a sacred bucket of bits that must never be modified, which is Linus's approach. The other is as a human-readable string, which is what HFS+ uses. They both have pros and cons. I personally agree with HFS+'s approach, but I understand that other people may decide that the bucket of bits approach is best.
The problem is, Linus couldn't even admit that any possibility other than "bucket of bits" could ever be correct in any situation. In fact, he couldn't even admit that human-readable string could be a desirable goal. He basically treated HFS+ as an incredibly bad implementation of "bucket of bits".
The problem is that the filenames are interpreted by computers, not by people. Your comment that filenames are "human-readable strings" is subtly changing the problem statement. You're conflating bits (interpreted by computers) with display (interpreted by humans).
They really are two fundamentally different things.
I've been working though similar i18n issues in the IETF. Various protocols started off as ASCII, and now need to support more. There have been 5+ years of heated discussion around the subject.
The consensus appears to be reaching the view that issues related to people are solely managed by the "human IO" portions of the computer. i.e. the display turns bits into glyphs, and the input turns keystrokes into normalized buckets of bits. Once the bits reach the rest of the system, everything else treats them as buckets of bits.
This approach means that the non-IO parts of the system are simple. Things like name comparisons are done via "memcmp". No i18n issues apply.
I'll go a little more extreme here, and say that IMHO this is the only approach which makes sense. Anything else is madness. Anything else means that your file system handles i18n.
You cannot simply put a "human" layer on top of the filesystem and expect it to work correctly or efficiently.
The most obvious case is, if I create two filenames that are identical except for normalization, how can the "human" layer possibly handle this?
Heck, even with just one filename it's trivial to produce braindead results. Assuming keyboard input is decomposed (I honestly forget), if I have a filename using composed characters, and I want to type this name into my terminal, I can't. And that's completely crazy to me as someone who works on user-facing software.
> if I create two filenames that are identical except for normalization, how can the "human" layer possibly handle this?
The "human IO" layer is responsible for normalization of strings sent to the rest of the system. Therefore, your scenario cannot occur in normal operations. If it does occur, too bad. Some part of the system allowed an illegal operation, and you now have two filenames which may display as identical, but which are stored as different "buckets of bits".
Your other scenario is the same. The issue of composed versus non-composed characters doesn't come up, as all input is normalized. If your human input layer can't normalize it, then tough. The system is broken.
What happens when you create a filename on a "new" system, and then mount it on an "old" system? What happens when you add new code points to unicode? What happens when you create new composition rules?
In your design, the answer is "oh, crap". The older file systems will make different decisions about name comparisons than a new file system. This leads to exactly the inconsistencies you're worried about.
In contrast, the IETF design means all file systems always make the same decisions. "memcmp" always returns the same results. Old systems cannot create file names using new code points, because their "display IO" layer forbids it. Again, if such a string are created, then the system created a string it can't understand. The system is broken.
It seems you have strong opinions here. These issues are being worked through in all internet protocols. The answer is not "fix every protocol to do string normalization and/or i18n".
Wait 5 years. You'll see this design take over every internet protocol, and (IMHO) every new file system. Nothing else is workable.
You're not making much sense. If all filenames that make it to the filesystem have passed through this layer, then this layer is effectively part of the filesystem. And since you say it shouldn't be part of the filesystem, then this must not be what you're trying to say.
If only filenames that are shown to/input by the human go through this layer, then you're fucked. You open the door to having all sorts of nasty filenames that the human cannot distinguish between, or that the human cannot even type. This is not good.
And I really don't get your point about new codepoints. Unicode is not going to introduce new composed codepoints. The set of composed codepoints that exist now is the same as the set of composed codepoints that will exist in 5 years.
It really seems like you're arguing for massive extra complexity that introduces all sorts of really bad edge cases simply so you can say "my filesystem preserves filenames", without actually getting any benefit of that because normal interaction is going to be normalizing those filenames anyway.
One of Linus's main points is that this is a layer problem.
Who cares if things get normalized in your Finder dialogs or other UX elements? But if something is being programmatically done to the filesystem itself (ie, by git via a write) it should do what is advertised -- write some bits. It seems wrong that the filesystem itself should be concerned with normalizing filenames or flipping bits of any kind.
It's not a layer problem. The fundamental problem is Linux is geared towards servers and HFS+ is geared towards consumers. On a server, it makes sense that whatever filename your program tries to write out is preserved perfectly. On a consumer-facing OS that's a terrible idea. Having mis-normalized filenames can cause all sorts of confusion. The most obvious is having two filenames that are grapheme-for-grapheme identical, but differ in the normalization of their characters. Less obvious is how to deal with situations where the user types a filename to open and their typed normalization differs from the actual normalization of the file on disk? Unless the filesystem itself performs the renormalization, there's no sane way to deal with this situation besides iterating over the directory and renormalizing each filename in the hopes of finding a match.
I don't follow, looking at the number of files on my laptop right now a very, very small percentage of them are intended for my interaction. The vast majority are intended purely for interaction by programs / the system. That tells me the file system should handle what is best for that case and my interaction should be proxied by another program that makes the file system fit for my consumption, e.g. Finder.
Of all the files on my system, the only ones that even hit the normalization issue are explicitly meant for human interaction. OS files typically aren't named using non-ASCII characters.
You seem to be deliberately ignoring the fact that they cannot be handled by a layer on top of the filesystem without introducing all sorts of weird behavior and severe inefficiencies.
medusa666, you appear to be hellbanned. All of your comments for the past 3 weeks are [dead]. I don't see anything obviously spammy in your history, although a karma of -2 is curious.
I don't understand the ins and outs of this website, or the scoring system. As far as I can tell, my score "went down" because I made an honest (but negative) observation about former world chess champion Gary Kasparov on some previous thread.
Without going into to much politics about how Linus handles his responses, how many random questions a day would you respond nicely to until you just started writing things like "because I am right!"?
In terms of file systems I believe that both opinions have flaws. Who thought that giving the user the choice of case sensitivity was a good idea when creating HFS+? And to take it a step further, why create any kind of hierarchy? why not just define a "bucket of bits, being content" with a set of associated meta for an inode that defines structure and naming, that way everyone can have their cake and eat it. Or am I missing something?
In reality there is no choice. If you choose case-sensitive you are basically fucked as there are loads of applications that will never run on your system. It is quite cynical of Apple really.
What's wrong with the choice? The vast majority of people won't even see the choice (since they're not formatting their drives manually in Disk Utility) and will get the case-insensitive default that Mac OS has always had. But there are occasional reasons why one would actually want a case-sensitive HFS+ filesystem and for those people, there's a choice.
25 comments
[ 2.8 ms ] story [ 53.8 ms ] threadCase in point, #6 there where he slams the HFS+ filesystem. This started an extremely long flamewar on the Git ML[1], between me and Linus, with everyone else piling in on the whole "Linus is always right" bandwagon. If you read the thread, you'll find that I was giving an explanation for why HFS+ stores filenames the way it does and why it actually makes sense in context, whereas Linus basically resorted to "it's crap because I say so". More specifically, he couldn't even admit that there were two approaches that had sensible arguments (filenames are human-readable strings, which HFS+ uses, vs filenames are sacred buckets of bits, which Linus believes). Instead, anyone who disagrees with him is obviously a moron.
[1]: http://thread.gmane.org/gmane.comp.version-control.git/70688
The problem is, Linus couldn't even admit that any possibility other than "bucket of bits" could ever be correct in any situation. In fact, he couldn't even admit that human-readable string could be a desirable goal. He basically treated HFS+ as an incredibly bad implementation of "bucket of bits".
They really are two fundamentally different things.
I've been working though similar i18n issues in the IETF. Various protocols started off as ASCII, and now need to support more. There have been 5+ years of heated discussion around the subject.
The consensus appears to be reaching the view that issues related to people are solely managed by the "human IO" portions of the computer. i.e. the display turns bits into glyphs, and the input turns keystrokes into normalized buckets of bits. Once the bits reach the rest of the system, everything else treats them as buckets of bits.
This approach means that the non-IO parts of the system are simple. Things like name comparisons are done via "memcmp". No i18n issues apply.
I'll go a little more extreme here, and say that IMHO this is the only approach which makes sense. Anything else is madness. Anything else means that your file system handles i18n.
The most obvious case is, if I create two filenames that are identical except for normalization, how can the "human" layer possibly handle this?
Heck, even with just one filename it's trivial to produce braindead results. Assuming keyboard input is decomposed (I honestly forget), if I have a filename using composed characters, and I want to type this name into my terminal, I can't. And that's completely crazy to me as someone who works on user-facing software.
The "human IO" layer is responsible for normalization of strings sent to the rest of the system. Therefore, your scenario cannot occur in normal operations. If it does occur, too bad. Some part of the system allowed an illegal operation, and you now have two filenames which may display as identical, but which are stored as different "buckets of bits".
Your other scenario is the same. The issue of composed versus non-composed characters doesn't come up, as all input is normalized. If your human input layer can't normalize it, then tough. The system is broken.
What happens when you create a filename on a "new" system, and then mount it on an "old" system? What happens when you add new code points to unicode? What happens when you create new composition rules?
In your design, the answer is "oh, crap". The older file systems will make different decisions about name comparisons than a new file system. This leads to exactly the inconsistencies you're worried about.
In contrast, the IETF design means all file systems always make the same decisions. "memcmp" always returns the same results. Old systems cannot create file names using new code points, because their "display IO" layer forbids it. Again, if such a string are created, then the system created a string it can't understand. The system is broken.
It seems you have strong opinions here. These issues are being worked through in all internet protocols. The answer is not "fix every protocol to do string normalization and/or i18n".
Wait 5 years. You'll see this design take over every internet protocol, and (IMHO) every new file system. Nothing else is workable.
If only filenames that are shown to/input by the human go through this layer, then you're fucked. You open the door to having all sorts of nasty filenames that the human cannot distinguish between, or that the human cannot even type. This is not good.
And I really don't get your point about new codepoints. Unicode is not going to introduce new composed codepoints. The set of composed codepoints that exist now is the same as the set of composed codepoints that will exist in 5 years.
It really seems like you're arguing for massive extra complexity that introduces all sorts of really bad edge cases simply so you can say "my filesystem preserves filenames", without actually getting any benefit of that because normal interaction is going to be normalizing those filenames anyway.
Who cares if things get normalized in your Finder dialogs or other UX elements? But if something is being programmatically done to the filesystem itself (ie, by git via a write) it should do what is advertised -- write some bits. It seems wrong that the filesystem itself should be concerned with normalizing filenames or flipping bits of any kind.
In terms of file systems I believe that both opinions have flaws. Who thought that giving the user the choice of case sensitivity was a good idea when creating HFS+? And to take it a step further, why create any kind of hierarchy? why not just define a "bucket of bits, being content" with a set of associated meta for an inode that defines structure and naming, that way everyone can have their cake and eat it. Or am I missing something?