Different bug AFAICT. The Twitter thread is talking about corruption happening from trying to open some path. Your link talks about trying to show a specific icon leading to corruption.
Moderators: Twitter is a terrible medium for long-format "articles". Can the main link be updated to this one, instead of the twitter-link?
Also: With twitter starting banning random accounts, like sci-hub, seemingly without any consistent justifications, shouldn't we reconsider twitter as valid source for HN news-stories?
Edit: seemingly different bug. Twitter is still all over terrible though.
Only problem with nitter.net is that 'this instance is over loaded' and so on. Nitter.net is running on some hetzner low instance and at least since beginning of 2021 nitter has said ' this instance is over loaded ' half the time you use nitter. So it goes
Counter case: Twitter forces authors to be succinct and to the point, and encourages them write in a style similar to the classical newspaper article structure where the most pertinent information is presented first
There is a wide variety of research published only on Twitter, precisely because the publishing process is so lightweight for writers, and every single paragraph can form the leaves of a single giant conversation. A high quality Twitter thread cannot easily be replicated using alternative media
You raise a good point, but an email on a mailing list lets you reply to specific sentences/paragraphs. I guess the fairly unique value that twitter is adding here is going from the specific paragraph to see if there are any replies.
Maybe webmention needs the ability to reference parts of a page, although I guess you could do that by linking to an anchor.
Commenting per paragraph has also been done by Medium. I'm not saying that Medium is good, it has its own problems. But that feature can be replicated outside Twitter/Mastodon/etc.
As long as you don't download anything, it should be fine? The last time I checked, browser caches don't use original filenames for storing cached objects on disk (like IE did in the old days).
I don’t think this is actually a corruption issue. I spent a lot of time with this yesterday analysing it in a VM and I couldn’t find any cases where on reboot it caused chkdsk to actually take any actions. It always found the file system clean.
What it seems to do is trigger a consistency check for the next reboot and some alarmist notifications. This is probably a simple check assertion somewhere which is seeing unexpected data passing through a call.
That is a problem but significantly less of one than actual corruption.
If anyone has evidence to the contrary and steps to reproduce it I’d be interested to try them.
I ran the test case and then chkdsk 5 times each on 1903, 2004 and 20H2 and it came back clean every time. It’s definitely possible I missed something but until it’s consistently reproducible then I’m on the fence.
One complaint is I’d expect something at the level of MFT corruption to trigger a bugcheck (panic). Doesn’t seem safe behaviour to let the system carry on with a corrupt MFT.
The article suggests a .url file could be found empty after the reboot. Perhaps the check assertion causes the filesystem to become read-only or discard the write cache or some other side effects that could lead to corruption?
If it's something along these lines, it may not be reproducible in a VM (where the host is responsible for waiting around for flushing delays)
That’s certainly a possibility. Will go and try it. The MFT writes are transactional in theory from my understanding and small files are stored in the MFT and shortcuts are small files.
Could be that the MFT is not flushed to disk after that event? That would suggest anything after the event would be broken. Also in line with my worries that this should be a bugcheck/panic rather than a soft assertion.
> I spent a lot of time with this yesterday analysing it in a VM and I couldn’t find any cases where on reboot it caused chkdsk to actually take any actions. It always found the file system clean.
That is very frequently the case with these kinds of filesystem bugs. You need actual hardware, not a VM. FWIW, it does seem to be reproducible on hardware.
I don't see why this bug would work any different in a VM than on a real bare-metal OS or disk?
It's an NTFS issue, i.e. a software issue. NTFS doesn't care if the underlying disk is virtual or bare metal. In fact, it kinda runs on virtual disks already even on bare metal, i.e. disk partitions (maybe even with a layer of encryption in between). There could be potential issues of course where you could get NTFS to trigger some hardware bugs, but that doesn't seem to be the case here, instead it is about accessing NTFS $I30 attributes.
I'm not a filesystem developer, but assuming the corruption is caused by a race condition in the filesystem driver the difference can come down to many aspects:
* Different timings accessing a raw device compared to a virtual drive.
* The presence of the host scheduler can also alter timings.
* CPU configuration on the guest that's not typical on a bare metal (single CPU VM?)
It's not at all obvious that a bug must manifest the same way on a VM than on bare metal, although probably more often than not that's the case. If filesystem testing is also done on VMs then that would explain how this slipped through QC.
The bug is, if you write garbage data into a file "name" similar to those holding filesystem internal metadata, the filesystem driver detects this and mistakenly thinks the file could be the real metadata, so it does a consistency check on it, and because it fails consistency check, it sets a "volume needs fsck" flag. Windows (after 18H2) then show an alert. The alert is extremely confusing (it literally says "disk corruption") to the user, hence all the buzz.
I believe you are correct. I could not see this cause filesystem corruption. From looking around in ntfs.sys, it just seem to set a 'this is corrupt' flags on that code path.
Does this affect windows-based web/file servers too? If a client can request an URL or similar with the pattern triggering a folder lookup on the server's NTFS volume?
Yes, but you wouldn't normally be able to access this file as a web client. Being able to view arbitrarily files as a web client is already a security concern.
The article says the path to the icon is cleared after a reboot, but what if it tries to access the icon first and fails to actually write the path to the shortcut (given that the filesystem is already hosed at that time). I wonder what happens if one creates the shortcut offline, e.g. using older MS-Windows, wine, reactOS or whatever. Then just copy the crafted lnk file to \Users\Public\Desktop and see what happens.
34 comments
[ 3.0 ms ] story [ 86.2 ms ] threadThe stuff about icons is just using a filetype to trigger the bug quickly by forcing windows to look up the path embedded in the icon.
Also: With twitter starting banning random accounts, like sci-hub, seemingly without any consistent justifications, shouldn't we reconsider twitter as valid source for HN news-stories?
Edit: seemingly different bug. Twitter is still all over terrible though.
There is a wide variety of research published only on Twitter, precisely because the publishing process is so lightweight for writers, and every single paragraph can form the leaves of a single giant conversation. A high quality Twitter thread cannot easily be replicated using alternative media
Maybe webmention needs the ability to reference parts of a page, although I guess you could do that by linking to an anchor.
Haven't found a way to run it on ZFS yet. There's a project that implements btrfs here: https://github.com/maharmstone/btrfs
What it seems to do is trigger a consistency check for the next reboot and some alarmist notifications. This is probably a simple check assertion somewhere which is seeing unexpected data passing through a call.
That is a problem but significantly less of one than actual corruption.
If anyone has evidence to the contrary and steps to reproduce it I’d be interested to try them.
One complaint is I’d expect something at the level of MFT corruption to trigger a bugcheck (panic). Doesn’t seem safe behaviour to let the system carry on with a corrupt MFT.
If it's something along these lines, it may not be reproducible in a VM (where the host is responsible for waiting around for flushing delays)
Could be that the MFT is not flushed to disk after that event? That would suggest anything after the event would be broken. Also in line with my worries that this should be a bugcheck/panic rather than a soft assertion.
That is very frequently the case with these kinds of filesystem bugs. You need actual hardware, not a VM. FWIW, it does seem to be reproducible on hardware.
This was a single test so the veracity of the outcome is somewhat debatable.
It's an NTFS issue, i.e. a software issue. NTFS doesn't care if the underlying disk is virtual or bare metal. In fact, it kinda runs on virtual disks already even on bare metal, i.e. disk partitions (maybe even with a layer of encryption in between). There could be potential issues of course where you could get NTFS to trigger some hardware bugs, but that doesn't seem to be the case here, instead it is about accessing NTFS $I30 attributes.
* Different timings accessing a raw device compared to a virtual drive.
* The presence of the host scheduler can also alter timings.
* CPU configuration on the guest that's not typical on a bare metal (single CPU VM?)
It's not at all obvious that a bug must manifest the same way on a VM than on bare metal, although probably more often than not that's the case. If filesystem testing is also done on VMs then that would explain how this slipped through QC.
This seems to be a deterministic bug tho. Open "special" file path to an $I30 attribute stream, crash and burn.
The bug is, if you write garbage data into a file "name" similar to those holding filesystem internal metadata, the filesystem driver detects this and mistakenly thinks the file could be the real metadata, so it does a consistency check on it, and because it fails consistency check, it sets a "volume needs fsck" flag. Windows (after 18H2) then show an alert. The alert is extremely confusing (it literally says "disk corruption") to the user, hence all the buzz.
Tested it three times. Two of the tests, chkdsk ran and Windows booted after.
Third time, after chkdsk it went into Windows startup repair, and ultimately led to a green screen where it wouldn't launch boot further.
Brought to you by the new Microsoft collective.
Also, it presents one of the rare cases where running chkdsk might actually do something useful.