It's interesting that both NTFS and APFS have now apparently been declared 'broken' due to performance problems with two very specific applications (Git and NPM), and that the reverse statement, that is's perhaps those applications that are being unreasonable, isn't quite as fashionable.
I mean, I don't use NPM, but I recently copied over some small-ish raw Git repositories between machines, and had several WTF moments when I looked at what was actually going over the wire.
Instead of refactoring two of the planet's most widely-used 'consumer' file systems, perhaps https://www.sqlite.org/appfileformat.html should be seen as a more reasonable way forward?
This got me curious and it seems that there is known APFS behaviors slowing things down on parallel I/O on same folders [1]
I would suspect that this is related to to the fact that reading/writing dirs have no atomic semantics in POSIX fs, and same directories are easily read/written simultaneously by unrelated processes, but surely is cause to edge cases bugs, especially in a world with backup system based on fs snapshots.
There is very low chance this is "something wrong with APFS on macOS". But there is something wrong for example with "Time Machine" backups reliability and breaking changes. Not supporting perfectly good hardware just to force users to buy new ones is also not nice. For example you can't easily move TM backup between disks or between hfs+ and apfs because time machine on hfs+ use hard links and on apfs snapshots. One time I've lost whole TM backup, not because of drive or filesystem failure but because of TM being unable to manage edge cases which arrise during network backup.
> There is very low chance this is "something wrong with APFS on macOS".
Why do you consider the chance of that to be low? APFS is functional and I trust to not eat my data, but it isn't a battle tested high performance file system that's done the rounds in the arena like, say, XFS.
I'm not sure if it's related but recently (last 2 years-ish?) I've noticed some issue with watching files on macOS. I edit a file but the dev server doesn't pick it up for way longer than it should. It's sometimes faster for me to restart the dev server than wait for it to notice the files changed.
I have not dug deeper into this aside from raising my open file limits and closing some apps I thought might be the culprit (but ended up not being the cause).
It's annoying because it's not consistent. The dev server will work just fine for a while then randomly slow down. It sucks because sometimes I spend a decent amount of time trying to figure out why my change/fix/etc isn't working only to realize I haven't been testing updated code (I made a code change and thought it hot-reloaded but it didn't).
File watching is flaky enough that polling based implementations are a better choice. You can stat tens of thousands of files every second, and easily optimize it to check frequently changes files more often. https://esbuild.github.io/api/#watch
Maybe it’s a combination of two things (a perfect storm):
1) APFS's snapshot feature, driven by Time Machine, adds a heavy tax on the massive number of file writes and deletes performed by pnpm and git.
2) macOS's less efficient handling of parallel I/O negates the speed benefits of modern, concurrent tooling, creating a bottleneck that is particularly severe when combined with the overhead from snapshots
11 comments
[ 3.1 ms ] story [ 24.8 ms ] threadI mean, I don't use NPM, but I recently copied over some small-ish raw Git repositories between machines, and had several WTF moments when I looked at what was actually going over the wire.
Instead of refactoring two of the planet's most widely-used 'consumer' file systems, perhaps https://www.sqlite.org/appfileformat.html should be seen as a more reasonable way forward?
I would suspect that this is related to to the fact that reading/writing dirs have no atomic semantics in POSIX fs, and same directories are easily read/written simultaneously by unrelated processes, but surely is cause to edge cases bugs, especially in a world with backup system based on fs snapshots.
This is a sad state of affairs.
https://gregoryszorc.com/blog/2018/10/29/global-kernel-locks...
Couldn’t these tests be instrumented to give a deeper dive into what may be happening here?
Why do you consider the chance of that to be low? APFS is functional and I trust to not eat my data, but it isn't a battle tested high performance file system that's done the rounds in the arena like, say, XFS.
I have not dug deeper into this aside from raising my open file limits and closing some apps I thought might be the culprit (but ended up not being the cause).
It's annoying because it's not consistent. The dev server will work just fine for a while then randomly slow down. It sucks because sometimes I spend a decent amount of time trying to figure out why my change/fix/etc isn't working only to realize I haven't been testing updated code (I made a code change and thought it hot-reloaded but it didn't).
1) APFS's snapshot feature, driven by Time Machine, adds a heavy tax on the massive number of file writes and deletes performed by pnpm and git.
2) macOS's less efficient handling of parallel I/O negates the speed benefits of modern, concurrent tooling, creating a bottleneck that is particularly severe when combined with the overhead from snapshots