> Back in 2009, Microsoft agreed to interoperability rules that provide third-party security apps with the same level of access to Windows that Microsoft gets. Microsoft agreed to provide kernel access in order to resolve multiple longstanding competition law issues in Europe.
> Apple has not been forced to make changes to how Macs work, but the European Commission has been targeting the closed nature of iOS, and Apple has warned that the updates that have already been implemented could lead to security risks in the future.
This is a valid and interesting comparison - Microsoft complied and Apple fought tooth and nail.
But also, it's kind of a moot point because absolutely no one is running Apple hardware at a flight kiosk.
If MS is really not allowed to protect their kernel, that part of whatever EU understanding they have should probably be revisited. I’m not sure it’s gonna hamper things legally too much to let MS, and everyone else for that matter, Apple, Linux, Android, whoever, erect defensive walls around their own kernels. This is just security 101 for devices these days, and it doesn’t stop anyone else from participating in the windows, android, apple ecosystem.
Not just that, but that would break backwards compatibility with a lot of niche but important apps, since Windows was very popular with use cases where companies have developed kernel space drivers for their Windows products. Yanking that away for security with break backwards compatibility.
Not even close - real time memory scanning? Suspicious memory access detection? Suspicious file access detection? Suspicious network traffic? Suspicious kernel calls? Introspection on running program code?
It’s already a major performance impact, throwing it into user space would kill those products entirely.
But misguided, as usual when someone defends Microsoft.
> If MS is really not allowed to protect their kernel, that part of whatever EU understanding they have should probably be revisited
They are allowed to protect “their kernel”. (And / or everything else, which AFAIK is what anti-virus software is supposed to do.) It's just that they're only allowed to do it the same way they allow everyone else to do the same thing.
So, being the lazy arseholes they are, they let everyone else into the kernel like they do it, in stead of doing what they should have done: Do it from userspace like anti-virus software used to work, or build a secure API that lets software protect the kernel without being able to take it down / prevent it from booting up — and use that API themselves, in their own separate anti-virus product.
Then that product would have been on an even footing in the marketplace with its competitors, which is what the agreement with the EU sought to achieve.
But Microsoft cheaped out and took the easier, cheaper — and riskier! — route. This debacle is as much on them as on Crowdstrike.
This is ridiculous though. Nothing is stopping Microsoft from offering “safe” access to both themselves and third parties.
Also, let’s not forget why these restrictions are applied in the first place - they kick in when the corp grows to become a gatekeeper, integrating multiple product categories and the expense of users and 3rd party vendors.
You can’t write programs like CrowdStrike falcon in user space on windows, because windows doesn’t offer the right kind of APIs and hooks from userspace to provide that kind of protection. On Mac OS CrowdStrike falcon runs in userspace already, and on Linux, the kernel interface is provided by eBPF, which acts as a sandbox, preventing certain classes of crash. So it really is true that Microsoft does not provide safe access like other operating systems do.
To be clear, they have to offer third parties the same access they enjoy themselves. If they provide a spell-checker that requires kernel implementation, they have to give all competing spell-checkers the same access.
Obviously, it seems ludicrous to implement spell-checking in the kernel. Imagine if MS imposed the discipline upon themselves to provide enough API that their own security products didn't reach for the kernel crutch.
Linux is different because it is open source. If a vendor wants to reach into the kernel, it's completely possible. If they want to provide patches to the kernel that allow their or any other product to do what it needs outside the kernel, they can. But even if the kernel provided all the necessary APIs, vendors could ignore those and insist on being inside the kernel. It is up to the user to use wisdom about what they install. And there is no unfair advantage to the OS vendor competing with third-party apps.
It is silly to blame regulators instead of the app vendor in this case. I'm not sure why MS feels the need to deflect blame. I know that some are eager to point the finger at MS, but MS shouldn't be so defensive. It makes it look like they think there is some merit to those arguments that has to be deflected.
> I'm not sure why MS feels the need to deflect blame.
Everyone is editorializing. Microsoft never actually blamed the EU. They just were explaining why their kernel was open to third parties in the first place.
Their kernel was open to SW devs since the very beginning, it's one of the reason that helped Windows grew in popularity especially as a video game platform. Early 3d games needed direct kernel access to extract maximum performance from the HW, but then MS built Direct3D as a wrapper for that direct HW access so games couldn't crash your whole system.
Correct. The issue was defender getting kernel access and nothing else getting that level of integration. Microsoft could have dropped defender to the same level as competing products, or dropped the product entirely. They chose to allow third parties access to kernel access instead.
Well, last point isn’t quite true. Alaska Airlines is primarily iOS. So, iPads scan your boarding pass and are used for printing luggage tags outside security, Flight attendants use iPhones on the planes, etc.
And they were of course mostly unaffected by the Cloudstrike outage.
Either way, I think it’s a strong point for Apple that antivirus is not required because the iOS security model protects against viruses pretty thoroughly. That’s just not the case for Windows most of the time. Also, do tech companies run antivirus on their linux-based containerized cloud workflows? Not really, the security model doesn’t require it. Points against windows server here
This is incorrect: Alaska Airlines wasn't a Crowdstrike customer. They use SentinalOne, a competitor. They absolutely use Windows desktop environments across their infrastructure.
CrowdStrike is deployed on Linux servers as well, so clearly companies like these do run EDR software on their Linux hosts.
Apple pretends they don't need antivirus the same way some Linux advocates will, but viruses exist on both platforms. Very few companies run their digital signage or internal application databases on macOS hosts, mostly because Apple stepped out of that market years ago.
Whenever Apple or Linux are deployed at the scale these CrowdStrike desktops are, you can assume similar software is deployed on any platform. This time it was a kernel crash, next time it could be MDM software locking all iPads out of all network access, or null routing all I/O requests in eBPF.
> Apple pretends they don't need antivirus the same way some Linux advocates will, but viruses exist on both platforms.
Sure. But when was the last time a company was in the news when they were hit with macOS-, iOS-, or Linux-based ransomware?
I'm in IT, but Windows was never my thing/niche. Generally I've viewed two problems with it:
1. when it becomes a monoculture where basically everything in the company runs on it
2. something about its architecture/designs appears (to me, at least) for very easy spreading of malware (does it have some weaker SSH-equivalent that allows easy remote control?)
At some level, we would have to accept that if we’re giving people, many of them hackers, access to kernel level facilities, there is a risk involved. If you want no risk, use a walled off OS. If you want a more flexible or permissive kernel architecture, then accept that the burden of securing it is kind of on you after a point.
Put in layman’s terms, I can put that riving knife on your table saw, but there’s still a risk to using it. It’s just a risky tool.
Guys, eBPF provides more limited access to kernel features. That’s by design. Whatever the platform, the idea with eBPF is to limit kernel access and provide safe access only where ‘necessary’.
The entire issue is that MS limited access to their kernel. If you want the people who use eBPF, windows developers, to have the same access to kernel features as the providers of eBPF facilities, MS themselves, then you’re effectively giving them kernel access.
There is no such thing as safe full featured kernel access.
I personally don't have problems with developers having kernel access. I can't imagine any open source OS fan objecting to this.
I do have problems with MS or any other vendor using kernel access unnecessarily. MS Paint should not run in the kernel. Just because you can doesn't mean you should.
The question is whether MS or any other vendor could provide the feature set their product enjoys without running in the kernel. If Windows offered anything like eBPF but MS security apps chose to run in-kernel instead of through eBPF, then MS should be forced to give competitors the same kernel access. If MS would play fair and restrict their non-OS teams to userspace they wouldn't have to play fair by providing access to kernelspace.
The argument is that it is impossible for MS non-OS teams to play in userspace because MS doesn't provide the APIs to make it possible. It seems that some other OSes might not suffer from this deficiency.
> The Falcon software was not able to wreak similar havoc on Macs because Apple does not give software makers kernel access. In macOS Catalina, which came out in 2020, Apple deprecated kernel extensions and transitioned to system extensions that run in a user space instead of at a kernel level. The change made Macs more stable and more secure, adding protection against unstable software updates like the one CrowdStrike pushed out. It is not possible for Macs to have a similar failure because of the change that Apple made.
What about Linux though?
Feels like this is just MS redirecting blame and using it as an opportunity to push the narrative that walled garden = good.
That piece of text you're quoting is mostly incorrect. Falcon didn't break Linux now because they haven't shipped a broken extension for Linux this time, only for Windows, but if you want to, you can definitely cause the Linux kernel to panic quite easily if you start inserting buggy drivers in kernel space like Crowdstrike was doing on Windows.
The primary one linked there is certainly not the same issue, it was a bug in the Linux kernel's ebpf handling. It happened to be triggered by Crowdstrike, but the bug is undeniably a Linux kernel bug which was subsequently patched, as ebpf programs should never be able to panic the kernel.
That's not to say that there haven't been other Crowdstrike fails on Linux, especially pre-eBPF module, but that's not one, and that class of failures has been eliminated in the move to the eBPF based module.
If they panic the kernel, they're presumably software that triggers those bugs in the kernel’s eBPF implementation mentioned in the comment you replied to?
I mean from a certain POV the Linux crashes were worse. Everybody understands that buggy kernel-mode drivers can bring down the OS and it's not the OS vendor's fault, but the Linux crashes were exactly what eBPF was supposed to make impossible.
Either someone is stupid enough to make this argument which I don't think is the case (The NT teams are crazy good), or they just want to use this opportunity to divert the blame, which is pathetic at best.
UAC, virtualization, hybrid kernel/user-space shenanigans, all were not in the OS at some point, and research and development, listening to other parties and taking inspiration from other OSes brought these advancements in security.
If Microsoft thinks offering kernel drivers for security (antivirus or otherwise) is a bad thing for the 3rd party companies, then by extension it is bad for any antiviral product they offer and they should absolutely find a new paradigm to securely implement them (eBPF like as some other folks suggested).
But saying "but apple does it !" is not a reasonable demande when your software runs respirators and nuclear facilities. (Apple are still cunts for having everything locked down but that's another conversation)
Microsoft didn't blame the European Commission for the outage (thats a ridiculous headline), they said they can't close off the Kernel due to agreements made with the European Commission. The original soruce macrumors is quoting is thie WSJ article, see last paragraph: https://archive.is/FLNKH
A Microsoft spokesman said it cannot legally wall off its operating system in the same way Apple does because of an understanding it reached with the European Commission following a complaint. In 2009, Microsoft agreed it would give makers of security software the same level of access to Windows that Microsoft gets.
They absolutely could close of the kernel - it's just that Defender as an antivirus product would have to use the same APIs. Apple doesn't offer a defender-like product, so the move was easier for them.
Not a kernel-level API - they'd move Defender to an API without kernel-level access. On Mac their API is called Endpoint Security Framework which lets antivirus monitor system calls without giving it kernel-level access. And System Integrity Protection is how they close the kernel. Microsoft would love to do the same, but also want their own Defender to have kernel-level access. The EU says they have to give third-party antivirus the same access they give their own antivirus, for anti-trust reasons. Personally I disagree with the EU here.
There is nothing technically stopping Microsoft from addressing this.
Fixing one’s kernel extensions does not violate anticompetitive laws. It does however make for click bait headlines if you as a corp choose to troll everyone affected by CrowdStrike instead of offering mitigations.
I find it unironic how for decades the tech community complained about the apple walled garden and how Microsoft isn't open about kernel internals and how it's hard to get things done with it compared to Linux which is open source. Now everyone is an EDR expert and they think Microsoft should be more like apple and less like Linux.
I think people don't know that Microsoft also has a Crowdstrike Falcon competitor that isn't too bad at all. While it would be funny to call out the hypocrisy in the tech world, I think the DoJ would indeed sue Microsoft unless it extracted it's own EDR and had it use the same API's it is asking other companies to use. Furthermore, there is something called the anti-malware scanner interface (AMSI) which Falcon and other EDRs indeed use and there is an ETW api that allows them to do additional monitoring as well. But that just isn't enough to counter many real world threats. In effect, the EDR must be like a rootkit to get complete system visibility and be able to stop any and all threats, kernel or userspace. Also, a user space process can prevent booting or freeze systems up.
There are multiple products called "Defender", I believe "Microsoft Defender for Endpoint" (which is an enterprise product, not the consumer Defender AV) would be considered the same.
I _think_ at least, the enterprise software space is confusing as hell :)
> Now everyone is an EDR expert and they think Microsoft should be more like apple and less like Linux
You are responding to a statement from Microsoft that compares its own behavior to that of Apple. Why do you characterize it as everyone else thinking Microsoft should be more like Apple?
Because Microsoft is responding to "everyone" who are asking it why it is allowing EDR makers to write drivers. Win10+ only load signed drivers, so every driver is approved by Microsoft. I'm criticizing the people and journalists that are asking this question. I'm sure congress will grill Microsoft's CEO as well and ask this same question again.
The alternative is that everyone in the Microsoft security ecosystem gets off kernel-mode drivers. Once ebpf-for-windows lands, it should be possible for Microsoft, CrowdStrike, and everyone else to run their filters in user-land. That puts everyone on a level playing field, and makes the ecosystem more secure overall.
I'm not affected by this (we don't use Windows) but something that's confused me: most of the various clouds have published advisories/status page updates for this.
Are they (e.g., MS) shipping VMs in their cloud (e.g., Azure) with CrowdStrike pre-installed? In which case I think people have a right to be upset with MS, as they've chosen an apparently poor quality vendor, and the EU argument seems like a complete distraction.
Or is the market of "audit checkbox checking security software" just such a monoculture that nigh every Windows VM out there was running this thing, but that it was installed by the owners of the VM (i.e., not by the cloud vendor), and now we see what happens when unfettered updates hit a monoculture? In which case, … I don't see how MS is to blame here; seems like you, the buyer of CrowdStrike, chose poorly. (And the EU thing is even more of a distraction.) (And I guess the cloud status page updates are just out of the goodness of the cloud vendors' hearts, or we don't think Windows sysadmins are competent enough to not blame their cloud, or both.)
61 comments
[ 3.3 ms ] story [ 126 ms ] thread> Apple has not been forced to make changes to how Macs work, but the European Commission has been targeting the closed nature of iOS, and Apple has warned that the updates that have already been implemented could lead to security risks in the future.
This is a valid and interesting comparison - Microsoft complied and Apple fought tooth and nail.
But also, it's kind of a moot point because absolutely no one is running Apple hardware at a flight kiosk.
If MS is really not allowed to protect their kernel, that part of whatever EU understanding they have should probably be revisited. I’m not sure it’s gonna hamper things legally too much to let MS, and everyone else for that matter, Apple, Linux, Android, whoever, erect defensive walls around their own kernels. This is just security 101 for devices these days, and it doesn’t stop anyone else from participating in the windows, android, apple ecosystem.
Those would have access of course, and hence have an extreme competitive advantage relative to other products. Even more than they do now.
Kiss any non-Microsoft antivirus or security software goodbye.
Not just that, but that would break backwards compatibility with a lot of niche but important apps, since Windows was very popular with use cases where companies have developed kernel space drivers for their Windows products. Yanking that away for security with break backwards compatibility.
It’s already a major performance impact, throwing it into user space would kill those products entirely.
But misguided, as usual when someone defends Microsoft.
> If MS is really not allowed to protect their kernel, that part of whatever EU understanding they have should probably be revisited
They are allowed to protect “their kernel”. (And / or everything else, which AFAIK is what anti-virus software is supposed to do.) It's just that they're only allowed to do it the same way they allow everyone else to do the same thing.
So, being the lazy arseholes they are, they let everyone else into the kernel like they do it, in stead of doing what they should have done: Do it from userspace like anti-virus software used to work, or build a secure API that lets software protect the kernel without being able to take it down / prevent it from booting up — and use that API themselves, in their own separate anti-virus product.
Then that product would have been on an even footing in the marketplace with its competitors, which is what the agreement with the EU sought to achieve.
But Microsoft cheaped out and took the easier, cheaper — and riskier! — route. This debacle is as much on them as on Crowdstrike.
Also, let’s not forget why these restrictions are applied in the first place - they kick in when the corp grows to become a gatekeeper, integrating multiple product categories and the expense of users and 3rd party vendors.
They are offering "safe" access, it's called userspace.
If anything, the Crowdstrike incident highlights the lack of diversity in the OS/security space.
Everyone using the same stack makes everyone vulnerable to the same bugs.
Obviously, it seems ludicrous to implement spell-checking in the kernel. Imagine if MS imposed the discipline upon themselves to provide enough API that their own security products didn't reach for the kernel crutch.
Linux is different because it is open source. If a vendor wants to reach into the kernel, it's completely possible. If they want to provide patches to the kernel that allow their or any other product to do what it needs outside the kernel, they can. But even if the kernel provided all the necessary APIs, vendors could ignore those and insist on being inside the kernel. It is up to the user to use wisdom about what they install. And there is no unfair advantage to the OS vendor competing with third-party apps.
It is silly to blame regulators instead of the app vendor in this case. I'm not sure why MS feels the need to deflect blame. I know that some are eager to point the finger at MS, but MS shouldn't be so defensive. It makes it look like they think there is some merit to those arguments that has to be deflected.
Everyone is editorializing. Microsoft never actually blamed the EU. They just were explaining why their kernel was open to third parties in the first place.
And they were of course mostly unaffected by the Cloudstrike outage.
Either way, I think it’s a strong point for Apple that antivirus is not required because the iOS security model protects against viruses pretty thoroughly. That’s just not the case for Windows most of the time. Also, do tech companies run antivirus on their linux-based containerized cloud workflows? Not really, the security model doesn’t require it. Points against windows server here
Apple pretends they don't need antivirus the same way some Linux advocates will, but viruses exist on both platforms. Very few companies run their digital signage or internal application databases on macOS hosts, mostly because Apple stepped out of that market years ago.
Whenever Apple or Linux are deployed at the scale these CrowdStrike desktops are, you can assume similar software is deployed on any platform. This time it was a kernel crash, next time it could be MDM software locking all iPads out of all network access, or null routing all I/O requests in eBPF.
Sure. But when was the last time a company was in the news when they were hit with macOS-, iOS-, or Linux-based ransomware?
I'm in IT, but Windows was never my thing/niche. Generally I've viewed two problems with it:
1. when it becomes a monoculture where basically everything in the company runs on it
2. something about its architecture/designs appears (to me, at least) for very easy spreading of malware (does it have some weaker SSH-equivalent that allows easy remote control?)
Just ask Maersk about these two points:
* https://www.wired.com/story/notpetya-cyberattack-ukraine-rus...
At some level, we would have to accept that if we’re giving people, many of them hackers, access to kernel level facilities, there is a risk involved. If you want no risk, use a walled off OS. If you want a more flexible or permissive kernel architecture, then accept that the burden of securing it is kind of on you after a point.
Put in layman’s terms, I can put that riving knife on your table saw, but there’s still a risk to using it. It’s just a risky tool.
Guys, eBPF provides more limited access to kernel features. That’s by design. Whatever the platform, the idea with eBPF is to limit kernel access and provide safe access only where ‘necessary’.
The entire issue is that MS limited access to their kernel. If you want the people who use eBPF, windows developers, to have the same access to kernel features as the providers of eBPF facilities, MS themselves, then you’re effectively giving them kernel access.
There is no such thing as safe full featured kernel access.
I do have problems with MS or any other vendor using kernel access unnecessarily. MS Paint should not run in the kernel. Just because you can doesn't mean you should.
The question is whether MS or any other vendor could provide the feature set their product enjoys without running in the kernel. If Windows offered anything like eBPF but MS security apps chose to run in-kernel instead of through eBPF, then MS should be forced to give competitors the same kernel access. If MS would play fair and restrict their non-OS teams to userspace they wouldn't have to play fair by providing access to kernelspace.
The argument is that it is impossible for MS non-OS teams to play in userspace because MS doesn't provide the APIs to make it possible. It seems that some other OSes might not suffer from this deficiency.
https://news.ycombinator.com/item?id=41029590
Microsoft points finger at the EU for not being able to lock down Windows
What about Linux though?
Feels like this is just MS redirecting blame and using it as an opportunity to push the narrative that walled garden = good.
https://www.theregister.com/2024/07/21/crowdstrike_linux_cra...
> mirashii 16 hours ago
The primary one linked there is certainly not the same issue, it was a bug in the Linux kernel's ebpf handling. It happened to be triggered by Crowdstrike, but the bug is undeniably a Linux kernel bug which was subsequently patched, as ebpf programs should never be able to panic the kernel.
That's not to say that there haven't been other Crowdstrike fails on Linux, especially pre-eBPF module, but that's not one, and that class of failures has been eliminated in the move to the eBPF based module.
Microsoft could create a similar safe Windows kernel API if they wanted.
True, but they did break Linux a few months back in much the same way they just broke Windows last Friday:
CrowdStrike broke Debian and Rocky Linux months ago, but no one noticed
https://www.neowin.net/news/crowdstrike-broke-debian-and-roc...
But since that break didn't ground a good percentage of global air traffic, it didn't get the same press coverage as this most recent breakage.
They can wreak all kinds of havoc with bad eBPF programs, but so far they haven't as far as I know.
UAC, virtualization, hybrid kernel/user-space shenanigans, all were not in the OS at some point, and research and development, listening to other parties and taking inspiration from other OSes brought these advancements in security.
If Microsoft thinks offering kernel drivers for security (antivirus or otherwise) is a bad thing for the 3rd party companies, then by extension it is bad for any antiviral product they offer and they should absolutely find a new paradigm to securely implement them (eBPF like as some other folks suggested).
But saying "but apple does it !" is not a reasonable demande when your software runs respirators and nuclear facilities. (Apple are still cunts for having everything locked down but that's another conversation)
More discussion: https://news.ycombinator.com/item?id=41029590
A Microsoft spokesman said it cannot legally wall off its operating system in the same way Apple does because of an understanding it reached with the European Commission following a complaint. In 2009, Microsoft agreed it would give makers of security software the same level of access to Windows that Microsoft gets.
Isn't that XProtect?
Fixing one’s kernel extensions does not violate anticompetitive laws. It does however make for click bait headlines if you as a corp choose to troll everyone affected by CrowdStrike instead of offering mitigations.
I think people don't know that Microsoft also has a Crowdstrike Falcon competitor that isn't too bad at all. While it would be funny to call out the hypocrisy in the tech world, I think the DoJ would indeed sue Microsoft unless it extracted it's own EDR and had it use the same API's it is asking other companies to use. Furthermore, there is something called the anti-malware scanner interface (AMSI) which Falcon and other EDRs indeed use and there is an ETW api that allows them to do additional monitoring as well. But that just isn't enough to counter many real world threats. In effect, the EDR must be like a rootkit to get complete system visibility and be able to stop any and all threats, kernel or userspace. Also, a user space process can prevent booting or freeze systems up.
Which one? Defender isn't a CS competitor.
I _think_ at least, the enterprise software space is confusing as hell :)
You are responding to a statement from Microsoft that compares its own behavior to that of Apple. Why do you characterize it as everyone else thinking Microsoft should be more like Apple?
https://github.com/microsoft/ebpf-for-windows
Are they (e.g., MS) shipping VMs in their cloud (e.g., Azure) with CrowdStrike pre-installed? In which case I think people have a right to be upset with MS, as they've chosen an apparently poor quality vendor, and the EU argument seems like a complete distraction.
Or is the market of "audit checkbox checking security software" just such a monoculture that nigh every Windows VM out there was running this thing, but that it was installed by the owners of the VM (i.e., not by the cloud vendor), and now we see what happens when unfettered updates hit a monoculture? In which case, … I don't see how MS is to blame here; seems like you, the buyer of CrowdStrike, chose poorly. (And the EU thing is even more of a distraction.) (And I guess the cloud status page updates are just out of the goodness of the cloud vendors' hearts, or we don't think Windows sysadmins are competent enough to not blame their cloud, or both.)