I can't imagine why anyone would want a case insensitive file system, the logic makes sense from some abstract humanitarian viewpoint, but it makes no sense whatsoever that different identities should be mapped/translated to mean the same..
a.txt != A.txt != a.TXT and so on. They're different names of different files.
Next generation *insensitive behavior should then be that cat.txt and neko.txt are the same (therefore only one of them allowed).
Filenames only exist for humans. I'm sure the machine would be plenty happy with just a number to identify each file. So if people prefer case insensitive names they should get them.
So then have another layer that shows insensitive things that point to files on the file system but leave the actual file system to simply map each unique identifier to a file.
Well, that is not true. POSIX API is fundamentally filename (and filehandle) based, access rights are path (and therefore filename) based, while inodes have flat (per filesystem) namespace. Inodes are mostly internal implementation detail that unfortunately leaked into some abstractions.
As a matter of API, yes. Conceptually, though, inodes are unique per filesystem, and names are not. Given symbolic links, even assumptions about path+filename are not always correct.
NTFS is case sensitive. The default is to turn it off. That is a historical accident from its DOS roots. It works decently until you somehow manage to get two files with different cases in there. Had to chase that a few times 'what do you mean Abc.txt has not been edited there I clearly see a abc.txt' Then findout you are editing one and looking at the other. So case insensitive makes natural sense for my brain. But it can be infuriating if the system has edge cases where it will do both. Chased one a few months ago in an npm repo. The thing had 2 files. On the dev box works fine (windows). Off the build machine (linux) not so much. One happened to pick the right file the other did not.
Technically, POSIX machine already identifies each file using a serial number. This defines a specific inode. File names are stored into directory entries which are just specific files mapping names to inodes. Multiple names can already point to the same inode.
So everything is determined by how you interpret directory entries.
And I can’t imagine why you want to be able to identify multiple different files by similar names where only casing differs. This is a completely arbitrary opinion and nothing fundamental.
> a.txt != A.txt != a.TXT and so on. They're different names of different files.
Only if you define your != operator that way. It’s completely reasonable to have equality under some normalization.
Yes, but then they have been made explicitly and they show up in a directory listing.
If you would make the filesystem case insensitive ls would only show a.txt, but suddenly a "virtual" A.txt a.Txt etc would also exist.
Now there are some folders on my machine that i would like to be case insensitive like ~/Downloads, never having to check if i downloaded distro.iso or Distro.iso would be nice. But i can see so many more problems with files i did not expect to exist and not showing up in my directory listing suddenly being there when i open something "else".
For developers, sure, we know the difference. But it's very hard to explain to your mother that MyFile.docx is a different file to Myfile.docx.
When I was in university this was the argument given by the lecturer of the operating systems course in favour of CI, and I found it convincing. (FTR he is the author of 33 internet standards, and not a fan of Windows in most other ways.)
Losing the ability to have two differently-cased filenames in the same directory doesn't seem like a great loss.
It's rather intuitive that you can have multiple files.
They're right there in your directory, and you see them:
a.txt
A.txt
And you get that they're files.. It's just plain weird to do extra work to eliminate this freedom.
I don't think anyone needs to explain it to anyone.. If you've already decided only to have files in one case, you'll never need to communicate this.. If you've decided to make use of both, you'll refer to them by their casing.. "mom, please open the myfile.txt, the one with an uppercase M", except, in writing, we don't even need to be so specific..
In my email to mom, I'll write "Please open MyFile.txt" and when mom sees theat there is a myFile.txt and MyFile.txt in the directory, she won't have trouble figuring out which one I meant."
By that argument, you might as well have files with the same name, if you can "just see them".
And let's not even start with non-English auditory where you have to render filenames as "С as a dollar, У, another С as a dollar, English T, E, M" (that's "system" btw).
I agree. I wish NATO alphabet could taught in schools and become the world standard for spelling the Latin alphabet; my quality of life improved significantly after I learnt it (by accident, on a job).
> there is a myFile.txt and MyFile.txt in the directory, she won't have trouble figuring out which one I meant.
Your mom is pretty good at IT, clearly.
In my experience, this sort of scenario with "normal users" will result in one of the following:
- user opens both files and tries to figure out which one "looks better" for the task at hand. If questioned later, they will likely hide this fact to avoid having to admit their judgement was somehow involved.
- user fires back an email saying "there are two files with that name !! Please do it for me, this is complicated !!11!"
Case sensitivity looks like an advantage only to us nerds. There is a reason many systems go all-uppercase-all-the-time, even where they could easily use casing: it's a big source of confusion. Spoken language has no case, after all.
> In my email to mom, I'll write "Please open MyFile.txt" and when mom sees theat there is a myFile.txt and MyFile.txt in the directory, she won't have trouble figuring out which one I meant."
But if she (or any normal user) only sees 'myfile.txt' or "myFile.txt" or "Myfile.txt", she wouldn't respond with "that file doesn't exist." Instead, she'd be more likely to assume you made a mistake in your specification.
Read through the comments on the LWN article: case insensitivity is a practical problem because user-space implementations (such as samba) must perform an O(N) check across all files in a directory before confirming that a file doesn't exist. At the same time, existing programs that assume case-insensitivity (coming from the Mac and Windows worlds, mostly) are often none too consistent about creating 'foo.txt' but opening it (potentially from another program) as 'FOO.TXT'.
> But it's very hard to explain to your mother that MyFile.docx is a different file to Myfile.docx
Why? Human languages are also case-sensitive (e.g. 'moon' and 'Moon' have different meanings).
Nowadays, casual users rarely enter filenames by name, and for output, case-insensitive filesystems are still case-preserving (and users expect that).
> Losing the ability to have two differently-cased filenames in the same directory doesn't seem like a great loss.
It is not really about this ability, it is about plenty of direct implementation and usability issues that it causes, like:
- it makes everything encoding-dependend (which complicates e.g. handling of mixed encoding filenames and filenames with invalid utf-8 sequences)
- interoperability with case-sensitive systems (unpacking archive containing files differing in case)
- backward compatibility (new unicode version adds new characters and their case mapping)
Considering that case-sensitivity is not the only such issue (there is also different unicode normalizations and homographs) it seems to me that it make sense to have FS code and low-level utils encoding-independent (and therefore case-insensitive) and do case sensitivity (and perhaps. homograph handling) only in high level user code.
> Why? Human languages are also case-sensitive (e.g. 'moon' and 'Moon' have different meanings).
You'd know I meant the same thing if I wrote "go to the moon" or "go to the Moon". If you wrote "Hi ian" and I replied "There's nobody here by that name, can't imagine who you might mean" you'd think I was being obtuse.
Humans by default make assumptions to error-correct. Computers don’t do that by default, but you could get bash (for example) to also make assumptions with a setting like:
set completion-ignore-case On
Personally, I’d prefer an obtuse computer over one that makes assumptions and is occasionally wrong.
Absolutely, but it's where I find people get tripped up who aren't as used to tech. The thing I have told the computer to do is technically wrong in a minor way but it's extremely obvious to anyone that looks at it what I want. There are multiple reasons why we usually want this to be the case (simpler to implement, cost of returning an error is low but the cost of automatically doing the wrong thing can be huge, etc).
The rules aren't universal either. Some machines are fine with different casing, some aren't. Email addresses aren't case sensitive, typically, but my user name might be. Email addresses have to be exact, except with gmail where . isn't important, and anything after + is ignored. Google search from the omnibox will assume what it thinks you wanted to search for by default but urls entered in the same place are extremely picky.
How about tonal marks (á, à, ạ, ã, ả) and "weird characters" (ơ,ư,â,ô) in languages like Vietnamese? Should the filesystem ignore those, too? After all, they are regularly left off in a lot of modern text based conversation.
If I have "bài tập #14.pdf" and "bai tap #14.pdf" in the same directory, basically everyone is going to expect them to be the same thing, not different things. Trying to explain they are different because filesystem developers only understand English-centric conventions isn't exactly easy to explain either.
Yes, I would quite like to see Linux filesystems require that file names be in valid UTF-8, with no new lines, at the very least. Normalisation would be a bonus but I'd want to do it a way that is helpful for compatibility with macOS and Windows.
That would definitely be a breaking change. I had some Python code that broke when running over the Boost source code because they include a test file in there that isn't valid UTF-8, so the files definitely exist. Admittedly that's a fairly contrived example (i.e. the deliberately didn't use UTF-8). Maybe it wouldn't be so bad in practice.
There are two ZFS options, “normalization” and “utf8only”, which are documented to do precisely this (https://www.freebsd.org/cgi/man.cgi?zfs). Never used it myself, though.
> Allowing multiple files with the same file name - differing just by case - is rather silly from a human point of view.
In the current paradigm where files are predominantly accessed by a path and filename, yes. But that need not be the only nor the best way of referring to persistently stored data.
It's definitely true that case-folding is generalisable to more generous equivalence notions - for example, more or less whitespace isn't a noticeable difference that people typically want two files for. People casually using their phones want to treat the filesystem as a search tool where fuzzy matches (across different directories and names) are reasonable responses to file system queries. There have been many attempts at it like WinFS, but that's a significant increase in scope from case-folding.
One incontrovertible justification for case-folding is to support Samba shares efficiently. Samba has to implement case-folding to clients and if it is not done in the filesystem datastructures you have to read in many directory entries to figure out if one would match with case-folding.
There is plenty of MS Windows and MacOS software that just expects case-folding from the operating system.
I'm not even sure case insensitivity would be the naive human thing to expect. When I first made contact with computers (and typewriters!) in elementary school around 1990, whenever I pushed space while holding shift I got worried that I created an "uppercase space" and quickly deleted it with backspace. I kept that habbit for quite a long time.
Having many files with same name, differing in case only (a cultural 'thing') is just a mess - there's no extra value here. Do you use case in spoken word ?
Arguments of performance or ease of implementation are technical details nobody cares (except devs of course :) )
We just end up living with this decission and there's no easy way out.
Can't you trivially solve this without addressing any of the 17 problems with case insensitivity by just not creating afile.txt and aFile.txt. In TB of data including files from the last 17 years I don't think I have any folders like that.
One of my NAS units ended up with a case-insensitive filesystem on it at some point, and I'm too lazy to go through the trouble of changing it. It's pretty annoying to do `mv filename.that.should.be.in.something.like.proper.case.mkv Filename.That.Should.Be.In.Something.Like.Proper.Case.mkv` and then realize that nothing actually changed.[0]
But my mother, case-sensitive filenames on her MacBook would make her life much worse. She can barely manage the high volume of files she creates as a teacher as it is, and the potential for accidentally doubling that number, or worse, is terrifying.
[0] So of course I have an alias `mvd filename.that.should.be.in.something.like.proper.case.mkv Filename.That.Should.Be.In.Something.Like.Proper.Case.mkv` that actually does `mv filename.that.should.be.in.something.like.proper.case.mkv Filename.That.Should.Be.In.Something.Like.Proper.Case.mkvzzz && mv Filename.That.Should.Be.In.Something.Like.Proper.Case.mkvzzz Filename.That.Should.Be.In.Something.Like.Proper.Case.mkv` and all is well.
Keep that stuff away from my filesystem is my first thought on the matter!
Having had a fair amount of experience dealing with case sensitive, case insensitive and HFS+ (argh!) file systems in rclone, all I can say is that it is really hard and much better left to user space. Rclone does syncing between diverse systems with different ideas about case sensitivity and normalization when matching up the the source file with the destination file.
I'm not even sure that I agree with normalizing the UTF-8 strings - some users really care that they have two files which look identical but have different encoding, for example files with é and é which evaluate to 0x65+0x301 and 0xE9 respectively but look identical.
I might possibly go for filenames having to be valid UTF-8 - that would at least be consistent with Windows and macOS.
PS I haven't found a way of discovering whether a file system is case insensitive or case sensitive without creating a file - is it possible?
For discovery, you could readdir, verify no case-sensitive overlaps, then stat a filename with changed case. Maybe compare inodes and number of hard links too if the changed name does succeed.
> I haven't found a way of discovering whether a file system is case insensitive or case sensitive without creating a file - is it possible?
You should be able to do ioctl(EXT4_IOC_GETFLAGS) on the directory and check if EXT4_CASEFOLD_FL is set. I'm not sure if this flag is applied recursively to child dentries (though I would assume it is).
I was assigned an Apple rMBP in 2012 and ended up installing Linux on it precisely because you couldn't trust the file system. In my case, I had a directory with a specific name in lower-case, and without realizing it, created a file with the same name in Pascal case which then wiped out my directory full of content. Back-ups are important but they shouldn't be required simply to rectify stupid OS decisions.
Obviously the filesystem does not delete a directory if you create a file with the same name. Either you did this yourself or some really dumb application did.
A few times I have checked out a git repo on my Mac that had two identically named files, only differing by case.
Talk about a confusing experience. As soon as you checkout the file, it tells you that a file has been deleted. All normal git ways of fixing the issue don't work, because it comes down to the git repo thinking there are two files but the file system not letting that happen.
I had a couple confusing situations happen in a Git repo, where a capitalized file name was changed to lowercase, or vice versa.
It caused an issue that only one of us in the team experienced, since they had a case-insensitive file system, I think Windows. The reverse also happened, where everyone but that person saw an issue, due to their case-sensitive file system.
In the end, "git mv" solved those cases, but I never fully understood how to avoid them occurring in the first place. I think we all just agreed on a convention of all lowercase folder/file names.
Case-insensitivity is language-dependent, people know about this, right?
Whether the lowercase version of "I" is "i", that depends on your language. Often it is, but for example in Turkish, lowercase "I" is "ı", and lowercase "İ" is "i".
How will the language be specified on the case-sensitive filesystem? Per file, per directory, per device?
Can I have English "I.txt" and Turkish "i.txt" in the same directory? Are they compared using the English rules or the Turkish rules?
What happens if I copy a file to a directory or device using a different language? To a device using a different filesystem that doesn't support languages? If I copy 10 files from a Turkish computer to a memory stick, then copy them to an English computer, will I get only 9 files at the end, because "I.txt" and "i.txt" got overwritten?
This one fact is (in my mind) the death-knell of this whole misguided idea.
Also note that once you set up the case sensitivity status on a directory/device, it could be hard to change it. For instance, if I have two files named "I.txt" and "i.txt" in a directory that is case-sensitive, I can't change that directory to case-insensitive using English rules. But I could using Turkish rules. You would have to do a full scan of the area you are planning to change in order to check whether anything would clash.
Also, you have the problem of filename searches. At the moment, it is possible to open a file by name in a directory that has lots of files, and the filesystem can do an indexed search for that file. If you change it to case-sensitive, then either you need to do a linear search instead (which could absolutely cripple some use-cases), or store an indexed case-insensitive lookup table, with the additional proviso that the index must be able to give you back the canonical filename. If the case-sensitivity changes, then this index must be recreated from scratch.
People are aware of it -- Unicode has casefolding rules which apply to all Unicode characters and are supposed to solve this problem, but I believe you can run into issues when the casefolding rules end up not using language-specific characters (meaning that any language information is lost in the translation).
Another example of this is the German eszett ("ß".upper() == "SS" but "SS".lower() == "ss" != "ß").
Because everything on windows used the win32 api applications are friendly to case insensitivity but you can still run into problems interacting with other systems that are not.
54 comments
[ 5.0 ms ] story [ 121 ms ] threada.txt != A.txt != a.TXT and so on. They're different names of different files.
Next generation *insensitive behavior should then be that cat.txt and neko.txt are the same (therefore only one of them allowed).
So everything is determined by how you interpret directory entries.
> a.txt != A.txt != a.TXT and so on. They're different names of different files.
Only if you define your != operator that way. It’s completely reasonable to have equality under some normalization.
Doesn't Linux/Unix already have such ambiguity? E.g., multiple hard links to the same file; symbolic links; etc.?
If you would make the filesystem case insensitive ls would only show a.txt, but suddenly a "virtual" A.txt a.Txt etc would also exist.
Now there are some folders on my machine that i would like to be case insensitive like ~/Downloads, never having to check if i downloaded distro.iso or Distro.iso would be nice. But i can see so many more problems with files i did not expect to exist and not showing up in my directory listing suddenly being there when i open something "else".
When I was in university this was the argument given by the lecturer of the operating systems course in favour of CI, and I found it convincing. (FTR he is the author of 33 internet standards, and not a fan of Windows in most other ways.)
Losing the ability to have two differently-cased filenames in the same directory doesn't seem like a great loss.
And let's not even start with non-English auditory where you have to render filenames as "С as a dollar, У, another С as a dollar, English T, E, M" (that's "system" btw).
Your mom is pretty good at IT, clearly.
In my experience, this sort of scenario with "normal users" will result in one of the following:
- user opens both files and tries to figure out which one "looks better" for the task at hand. If questioned later, they will likely hide this fact to avoid having to admit their judgement was somehow involved.
- user fires back an email saying "there are two files with that name !! Please do it for me, this is complicated !!11!"
Case sensitivity looks like an advantage only to us nerds. There is a reason many systems go all-uppercase-all-the-time, even where they could easily use casing: it's a big source of confusion. Spoken language has no case, after all.
But if she (or any normal user) only sees 'myfile.txt' or "myFile.txt" or "Myfile.txt", she wouldn't respond with "that file doesn't exist." Instead, she'd be more likely to assume you made a mistake in your specification.
Read through the comments on the LWN article: case insensitivity is a practical problem because user-space implementations (such as samba) must perform an O(N) check across all files in a directory before confirming that a file doesn't exist. At the same time, existing programs that assume case-insensitivity (coming from the Mac and Windows worlds, mostly) are often none too consistent about creating 'foo.txt' but opening it (potentially from another program) as 'FOO.TXT'.
Why? Human languages are also case-sensitive (e.g. 'moon' and 'Moon' have different meanings).
Nowadays, casual users rarely enter filenames by name, and for output, case-insensitive filesystems are still case-preserving (and users expect that).
> Losing the ability to have two differently-cased filenames in the same directory doesn't seem like a great loss.
It is not really about this ability, it is about plenty of direct implementation and usability issues that it causes, like:
- it makes everything encoding-dependend (which complicates e.g. handling of mixed encoding filenames and filenames with invalid utf-8 sequences)
- interoperability with case-sensitive systems (unpacking archive containing files differing in case)
- backward compatibility (new unicode version adds new characters and their case mapping)
Considering that case-sensitivity is not the only such issue (there is also different unicode normalizations and homographs) it seems to me that it make sense to have FS code and low-level utils encoding-independent (and therefore case-insensitive) and do case sensitivity (and perhaps. homograph handling) only in high level user code.
You'd know I meant the same thing if I wrote "go to the moon" or "go to the Moon". If you wrote "Hi ian" and I replied "There's nobody here by that name, can't imagine who you might mean" you'd think I was being obtuse.
set completion-ignore-case On
Personally, I’d prefer an obtuse computer over one that makes assumptions and is occasionally wrong.
Absolutely, but it's where I find people get tripped up who aren't as used to tech. The thing I have told the computer to do is technically wrong in a minor way but it's extremely obvious to anyone that looks at it what I want. There are multiple reasons why we usually want this to be the case (simpler to implement, cost of returning an error is low but the cost of automatically doing the wrong thing can be huge, etc).
The rules aren't universal either. Some machines are fine with different casing, some aren't. Email addresses aren't case sensitive, typically, but my user name might be. Email addresses have to be exact, except with gmail where . isn't important, and anything after + is ignored. Google search from the omnibox will assume what it thinks you wanted to search for by default but urls entered in the same place are extremely picky.
If I have "bài tập #14.pdf" and "bai tap #14.pdf" in the same directory, basically everyone is going to expect them to be the same thing, not different things. Trying to explain they are different because filesystem developers only understand English-centric conventions isn't exactly easy to explain either.
There are bigger problems, though - currently Linux allows having multiple files with the exact same file name, differing only by UTF-8 normalisation.
Having a unique name is uncommon.
In the current paradigm where files are predominantly accessed by a path and filename, yes. But that need not be the only nor the best way of referring to persistently stored data.
One incontrovertible justification for case-folding is to support Samba shares efficiently. Samba has to implement case-folding to clients and if it is not done in the filesystem datastructures you have to read in many directory entries to figure out if one would match with case-folding.
There is plenty of MS Windows and MacOS software that just expects case-folding from the operating system.
The article is old and this was released in 5.2. It can be turned on per (empty) directory with chattr https://kernelnewbies.org/Linux_5.2#Optional_support_for_cas...
But my mother, case-sensitive filenames on her MacBook would make her life much worse. She can barely manage the high volume of files she creates as a teacher as it is, and the potential for accidentally doubling that number, or worse, is terrifying.
[0] So of course I have an alias `mvd filename.that.should.be.in.something.like.proper.case.mkv Filename.That.Should.Be.In.Something.Like.Proper.Case.mkv` that actually does `mv filename.that.should.be.in.something.like.proper.case.mkv Filename.That.Should.Be.In.Something.Like.Proper.Case.mkvzzz && mv Filename.That.Should.Be.In.Something.Like.Proper.Case.mkvzzz Filename.That.Should.Be.In.Something.Like.Proper.Case.mkv` and all is well.
Having had a fair amount of experience dealing with case sensitive, case insensitive and HFS+ (argh!) file systems in rclone, all I can say is that it is really hard and much better left to user space. Rclone does syncing between diverse systems with different ideas about case sensitivity and normalization when matching up the the source file with the destination file.
I'm not even sure that I agree with normalizing the UTF-8 strings - some users really care that they have two files which look identical but have different encoding, for example files with é and é which evaluate to 0x65+0x301 and 0xE9 respectively but look identical.
I might possibly go for filenames having to be valid UTF-8 - that would at least be consistent with Windows and macOS.
PS I haven't found a way of discovering whether a file system is case insensitive or case sensitive without creating a file - is it possible?
You should be able to do ioctl(EXT4_IOC_GETFLAGS) on the directory and check if EXT4_CASEFOLD_FL is set. I'm not sure if this flag is applied recursively to child dentries (though I would assume it is).
Talk about a confusing experience. As soon as you checkout the file, it tells you that a file has been deleted. All normal git ways of fixing the issue don't work, because it comes down to the git repo thinking there are two files but the file system not letting that happen.
It caused an issue that only one of us in the team experienced, since they had a case-insensitive file system, I think Windows. The reverse also happened, where everyone but that person saw an issue, due to their case-sensitive file system.
In the end, "git mv" solved those cases, but I never fully understood how to avoid them occurring in the first place. I think we all just agreed on a convention of all lowercase folder/file names.
Whether the lowercase version of "I" is "i", that depends on your language. Often it is, but for example in Turkish, lowercase "I" is "ı", and lowercase "İ" is "i".
How will the language be specified on the case-sensitive filesystem? Per file, per directory, per device?
Can I have English "I.txt" and Turkish "i.txt" in the same directory? Are they compared using the English rules or the Turkish rules?
What happens if I copy a file to a directory or device using a different language? To a device using a different filesystem that doesn't support languages? If I copy 10 files from a Turkish computer to a memory stick, then copy them to an English computer, will I get only 9 files at the end, because "I.txt" and "i.txt" got overwritten?
Also note that once you set up the case sensitivity status on a directory/device, it could be hard to change it. For instance, if I have two files named "I.txt" and "i.txt" in a directory that is case-sensitive, I can't change that directory to case-insensitive using English rules. But I could using Turkish rules. You would have to do a full scan of the area you are planning to change in order to check whether anything would clash.
Also, you have the problem of filename searches. At the moment, it is possible to open a file by name in a directory that has lots of files, and the filesystem can do an indexed search for that file. If you change it to case-sensitive, then either you need to do a linear search instead (which could absolutely cripple some use-cases), or store an indexed case-insensitive lookup table, with the additional proviso that the index must be able to give you back the canonical filename. If the case-sensitivity changes, then this index must be recreated from scratch.
Another example of this is the German eszett ("ß".upper() == "SS" but "SS".lower() == "ss" != "ß").
(Reading this discussion thread feels like some strange alternative reality where "take ... from my cold dead hands" is transported to FS world.)
https://docs.microsoft.com/en-us/windows/win32/api/fileapi/n...
Because everything on windows used the win32 api applications are friendly to case insensitivity but you can still run into problems interacting with other systems that are not.
https://www.hanselman.com/blog/GitIsCasesensitiveAndYourFile...