This is the part of open-source that bugs me. There is and likely always will be some level of politics in play.
There seems to a precedent here and people are able to get around it by who they know or their reputation in the community.
I’ve seen similar things happen in the CNCF.
It’s understandable but still rubs me the wrong way.
On good definition of politics is "the set of activities that are associated with making decisions in groups, or other forms of power relations between individuals, such as the distribution of resources."
All collaborative projects are inherently political.
But that's obviously not what people mean when they criticize something as being "political". The negative connotations of the word include people having double standards based on identities or group affiliations, being disingenuous about one's actual values and motivations, abusing relationships and partnerships to further an often hidden agenda, etc.
Most of the time I see people complaining, it's more from 'I don't understand why I have to abide by the community's values (plural) in order to submit code or otherwise get free labor out of them.'
Well take, for instance, Microsoft's reputation of embrace, extend, extinguish. That's a perfect example. I don't think they're that bad any longer, but I used to work with a lot of Apache projects where people would vote against committers from another company that they happened to compete with, and have pretty flimsy explanations for why they didn't think they belonged in the community. Or people would refuse to engage on design discussions and then shoot down proposals - usually just on opposite sides of some corporate competition. That's beyond free labor - it's mixing corporate interests with community project governance, and it's contrary to what should be the inclusive and good-willed nature of non-profit open-source projects like that.
> There seems to a precedent here and people are able to get around it by who they know or their reputation in the community.
Well, reputation matters.
Greg Kroah-Hartman let the Habana Labs stuff in over objections, but Habana actually open sourced its libraries later. Qualcomm has demonstrated repeatedly that they will never do that.
And, personally, I'm kinda glad GKH cut Habana some slack. A nice open-source library now exists for that and GKH cutting Habana slack may have contributed to the process.
GKH has also pointed out that he has changed his mind and wouldn't do that again.
People will often do something once because they see it as an exception. Now that this kind of thing is no longer an "exception", people are looking at it differently.
> There is a trend toward devices that are programmable from user space and the kernel is simply used as a conduit to carry proprietary code and data between user space and the device; only the device maker has any real view into what is actually happening inside it.
They just keep coming up with new ways to take away the user's control...
Every bit of control taken away from the user is just another avenue that can be used to make money from them.
More than that though, it's a way to ensure only select people can be creators, the rest must be delegated to consumers.
Computers as they've been for the last few decades has enabled more people to create so many things than any other time in history. For one of the first times in history, the last 30 years or so has allowed any single person on this planet, with enough money for a computer to do amazing things.
But, this upsets the status quo. Businesses and I guess the 'powers' for lack of a better word, that profited off society as it was have been threatened and in many cases overturned in a short time.
However, they're catching up faster than the average person and through a combination of marketing and just market control, companies are closing hardware platforms faster than what it's taken for most people to begin to utilize all this amazing technology.
And it's sad, I know my kids won't have the experience of booting up an atari computer and laughing as they use their first loop to make a bunch of random things appear on their screen over and over. They won't have that experience cobbling computers together from spare parts and getting on the internet for the first time and just being amazed at all the things out there. They'll never get that feeling of really being free to do anything because they have a computer.
As a kid, even when everything else was really shitty, being able to just turn on a computer and do anything, whether I was supposed to or not, just really helped me through so much and honestly, I don't know how my life would have turned out if I couldn't have done that.
By the time I have kids and they're old enough, unless I buy some retro computers and set it up or something, everything they end up using or seeing will be a locked down device the sole purpose of which is to constantly track them and market to them. The internet is corporate, sanitized, and locked away in these little bubbles provided by platforms.
So much has been lost and slowly but surely, the ability to just pick up a computer and create anything we want is being stripped away by hardware vendors and these huge dominating platforms that control every aspect of the computing experience and feel the need to profit off of and dictate everyone's computer use.
Why would you want to buy them a retro computer? The reality would look closer to something like Windows S being installed on every PC and you'd have to shell out $2000 for an expensive Windows license to be able to run unsigned applications. Also, the raspberry pi is pretty much the epitome of giving children access to a programmable computer.
My personal fear is that people will think that they have to explicitly decide to invest into becoming a software developer by buying developer tools the same way you have to invest into dedicated wood working tools to become a carpenter.
It's already happening. Look at the tablet/mobile market. These devices are easily capable of providing a development environment but it's essentially impossible to use them like that.
The IPad Pro being the worst of the worst. It even markets itself as a "real" computer.
The drivers are probably riddled with security bugs. For the at-home consumer security through diversity and obscurity is sufficient, their adversary is not the government but malware authors. Malware authors can and do use source code as their starting point for exploits. Especially vendored known-bad code, the main thing that is hiding in these drivers.
The manufacturer’s adversary isn’t consumers. It’s not even other small time manufacturers making knockoffs. It’s Google, Microsoft, Amazon and Apple, who are obviously the biggest violators of the spirit, technological and economic, of open source.
As soon as you open source your pathetic network driver, if it happens to benchmark the best in some niche way a Google team needs, it’s not like they go and buy your hardware. It’s just some guy who looks at your source and copies-via-rewriting it internally at Google for the hardware they already own and have source access to. Like how are you going to prove that copyright violation happens? People can barely get GPL judgements against random, low money defendants.
Look at how many businesses they are in. Google is making WiFi routers. Apple makes its own GPUs now. What will they start with? Open source code. It’s not just the AppGet guy, or some niche feature. It’s literally everything, top to bottom. It is the core of the trillion dollar market cap culture.
Nobody has or will ever succeed in a source code copying lawsuit against Google. Apple is going out there and using your shit until they copy it too. Your chips, your wireless stack, your graphics stack, every little thing. They will copy it. Amazon and Microsoft have far flung offices loaded up with people just violating the spirit of open source and legal force of EULAs via copying all day. That is how some guy survives, gets enough work for a year that matters to the bottom line of a multi billion dollar corporation, far out of sight of the compliance culture at HQ.
Every giant company does this. Engineers make careers out of copying open source stuff into some other language, making it seem like it’s clean room implemented when of course it is not. They are the adversaries, not users.
On the point about riddled with bugs, the Qualcomm proprietary drivers absolutely are riddled with them. Even in their closed source and proprietary incarnation, take a skim through Google's monthly android security bulletins, specifically for the Qualcomm closed source components.
It's a long time since I did it, but I analysed the frequency and severity of identified vulnerabilities, looking at QCOM open and closed components... The results were fairly easy to predict and guess - the closed components had many more critical vulnerabilities, occuring at a far greater rate.
One reason Qualcomm wants to keep these closed is because of IP and the fact they often buy in third party IP (especially in imaging for example). The other reason is because they sell their proprietary userspace drivers to manufacturers as a "board support package", and they need to select the length of time they want updates for. In the old days before treble, when Google changed the HAL every release of android, this meant not being able to run using the old binaries, or having to write shim drivers to wrap old (closed, but exported) function calls with the new expected ones.
Post-Treble, the Swiss cheese nature of their driver code means OEMs need these updates to avoid their phones being compromised, ensuring further recurring revenue.
The best thing that could happen for Android security would be the opening of the entire Qualcomm userspace drivers, so they could be maintained by some competent developers, and eventually unified into an equivalent project akin to the mainline kernel, so that common code could be shared between generations, and therefore security patches can be applied to past devices' components.
And then the next best thing would be for QCOM to open source their modem and RIL (Radio interface layer), because both of those are vulnerability ridden messes of legacy code - I believe their radio baseband firmware is compiled using an utterly ancient compiler still, and lacks even basic secure coding principles. Occasionally you'll see some black box vulnerability research looking into these basebands and they tend to validate the above findings quite nicely.
While most people probably agree that the full open sourcing is the best for the consumer, eventually somebody needs to be paid to do the work.
It's easy to say in hindsight where Android is massively successful, Google will someday allocate resources to recreate Qualcomm, from scratch, avoid patent issues like the consortia did with VC1, and expire the SoC industry. However that seems hard, also Apple has tried this and still hasn't finished an alternative to Qualcomm.
Imagine it was 2006, when there was no one successful to copy. Even if Google were as rich then as it is today, but had the knowledge that Android was definitely going to be a hit, it would definitely not become a phone SoC company - neither did Microsoft nor Blackberry (did Nokia? I honestly don't know). They had the same conviction then in its success, so it didn't matter how much foresight about security issues affecting millions of people there would have been. In any circumstance, they would have not given the kit-and-kaboodle away for free in open source. They'd feel, much like Qualcomm feels today, that they designed something original. Oh, and so now it cannot be shared.
Qualcomm got there first and sells this thing, that's the simplest reason they will never open source. They could achieve all your goals in a difference of execution, it's got nothing to do with the license of their source code. People have written secure closed source software.
For me, there's an important difference between Linus Torvalds copying Unix to make Linux and some giant company copying some other giant company's stuff. Everyone is dancing around the fact that the two are not equivalent and pretending that they are. Someone eventually has to write original drivers that are good, for complex hardware. That person or entity has to get paid somehow. We're being really naive if we just assume that Google, an advertising company that in recent history never starts something unless there is already ample market success to follow (or displace), would do a good job, just because its engineers are highly paid, have lots of time and do algorithms questions well.
>eventually somebody needs to be paid to do the work
>That person or entity has to get paid somehow
In my experience, it is very hard to get companies to even admit what the real cost of open sourcing something like this is. Lots of them just don't even want to talk about it or show their estimates at all, they just say "we will never open source" like Qualcomm does and the conversation goes nowhere. Even if the price is very high and it takes many years, we still need to start somewhere. Then the industry as a whole can chip away at it slowly, because even very slow progress is better than nothing at all. And what's even worse than no progress is if we just give up and sit around waiting for some other company like Google or Apple to foot 100% of the bill to redo all the work over again from scratch.
I think part of the unwillingness is due to the incorporation of external third party IP they realise they have no real prospect of being able to get a license to open source. In some areas like imaging this always seemed to be particularly problematic - from memory, some of the early "Sony Mobile" Android devices (thinking Xperia Z, Z2 era, etc.) actually used TrustZone DRM to prevent use of certain imaging libraries they had "licensed in", in order to discourage/prevent reverse engineering of them by users that had unlocked the bootloader and thus would likely have root access to facilitate meddling with them.
I believe that people eventually removed the TrustZone checks, but the intent was clear. This isn't Qualcomm directly though, but I imagine it shows part of the root of the challenge here.
Yes, the Freedreno work is a really good start, and an admirable one at that which has reached fruition!
I think there's quite a lot of closed userspace drivers that need (or would benefit from being) freed - I don't have a list to hand, but a quick look inside the /vendor partition (and directory) of a recent phone will give an idea of the closed userspace drivers being shipped on devices.
That's unfortunate. I remember 7 years ago I specifically seeked out an Atheros-bassed WiFi card for my computer because of Linux support, and more recently purchased a high-end router based on OpenWRT support. And a few months ago I specifically got a $450 AMD GPU because Nvidia doesn't like free drivers. Seems like Intel is the only option left for networking...
Qualcomm being closed source is the biggest issue with running good (user-respecting) software on phones, since basically every single Android has a Qualcomm processor/GPU. Samsung has some Exynos stuff but not in the USA. Only because of that, I'm looking into the PinePhone and Librem 5, which can run the mainline kernels.
I realize I'm far from a normal user, but the amount of ill will these companies develop is pretty massive. Broadcom for example has been permanently associated with 'this will be a pain in the ass to get working, and will break your system on updates.' And when I was buying the motherboard / wifi card / wifi router recently I immediately disqualified anything that had a Broadcom chip or didn't specify a chip.
For consumer sales, technologically aware people like us have an amplified impact because people trust us for recommendations. The AMD GPU purchase doubled to $900 because I suggested it to my brother when he needed a new one last month, even though he uses Windows. I've heard this also accredited to the rise of services like Google & GMail; 'techies' used them and their influence on family & friends was a big amplifier to help get users. So I wonder why Qualcomm and Nvidia are so hostile.
> Qualcomm being closed source is the biggest issue with running good (user-respecting) software on phones, since basically every single Android has a Qualcomm processor/GPU.
Actually there are kernel drivers for several Snapdragon platforms now, to the point that a Poco F1 can boot both GNU and AOSP-based systems off mainline.
Strong support for rejecting kernel drivers with a proprietary userspace counterpart. Totally violates the spirit of the GPL. I was NACK to this WSL driver, too, totally flies in the face of the rest of the Linux graphics stack. Companies should get in the habit of talking to the kernel early, and not just dumping a patchset over the fence and hoping we like it.
Why do companies care if their driver is in the main kernel repository or not? All that matters is if distributions ship it, which they have to if people want their computers to work. Why don't they just say 'take it or leave it.'
The kernel has no stable interface for device drivers and in fact internal APIs routinely change. If your device driver is in the kernel repository, it will be updated by the people making changes and you get the chance to give feedback.
But again RedHat and Canonical for example can't ship without updating the changed kernel code to match the driver, so can't they leave it to them to fix?
Red Hat doesn't ship any proprietary module. However, they do list a number of symbols and struct definitions as stable, unlike the upstream kernel, and preserve them for the life of a RHEL major release. This is why RHEL stays on an old kernel release and backports new features to the old release, instead of releasing new kernels.
Note that most of the GPU driver problems are on phones and neither RHEL nor Ubuntu officially support phones at all. And there are also many PC components that RHEL/Ubuntu don't support either, although usually that's just because the hardware is too new.
Red Hat ships nouveau. Nvidia is the one that chooses which distros to support (in addition to upstream kernels) and adapts their driver shim if needed.
Your mistake is thinking RedHat and Canonical are some sort of players here. The vast vast majority of Linux deployments are not from any sort of Linux distribution company.
Nvidia makes way more money from Quadro and Tesla cards than from the consumer cards (probably in general, and certainly running on Linux). "Company-made" Linux (including CentOS) is absolutely a player there.
Companies would then have to maintain the drivers themselves. If I remember correctly, the Linux kernel gets over 20 patches per hour. They have to pay developers to maintain the out-of-tree drivers or get left behind.
This is one reason why so many mobile devices run ancient kernels which can't be updated. They stopped maintaining the out-of-tree drivers so people remain permanently stuck on the latest version that still works.
Good question. The answer is that it requires you to keep your driver maintained as a patch that applies against the kernel you want to use it on.
Every release, there's refactoring and improvements of the kernel subsystems. They don't offer stable APIs (by design) for drivers - you need to keep it up to date.
If you want to see this for real, try getting a "code thrown over the wall once" type embedded device kernel tree, figure out the kernel version it shipped on (likely ancient), and try to apply the latest patch level for that release to it. Then resolve the inevitable issues even just applying a few minor bug fixes.
Now if you to to update to the next major kernel version, you'll start to find functions your driver tried to call aren't there, or have new function definitions. You need to dig through the code to see what's changed. Over a few major versions, you'll eventually realise this is a whole lot of bother. If you just mainline your driver, you'll no longer need to worry about this costly maintenance burden - it might well be a few engineers spending most of their time on it. That quickly adds up, and suddenly mainlining starts to make sense.
Nvidia has to keep updating their proprietary driver every release, and presumably has a few developers doing this. I can't speak for how well it works, as I always go for AMD, but other comments here seem to confirm distributions will happily break Nvidia drivers because they aren't mainline and therefore aren't tested.
Yup, and with that, welcome to the world of semi-proprietary ARM development boards. Leave security patches at the door, and enjoy your version 3.5 kernel for as long as the hardware lasts!
Joking aside, I think a lot of this comes down to the incentives (or lack thereof) in the ecosystem - there is no real penalty for releasing an insecure product, or one that can't be kept up to date. Thus inventivising shipping on an outdated kernel you can't maintain, and never patching it.
If those shipping products had to factor in the externalities of this decision, they would rapidly be mainlining everything to reduce the costs of this. The cost of "mainlining" their code will be trivial compared to the (unmeasured) cost of the disposal of the equipment during a reasonable lifespan due to lacking updates, and the replacement of the equipment.
> They don't offer stable APIs (by design) for drivers
The real problem lies in this practice. Which other popular system uses this approach? Windows, macOS but also open source alternative like FreeBSD all have well-defined kernel API and drivers SDK. So, not providing a standard interface but then crying for the lack of third party drivers is hypocritical: the problem is self-inflicted.
Good link - there's also a clear case that "requiring" the kernel to provide long-term stable driver API/ABIs is simply reallocating a cost towards the benevolent open source maintainers, to allow closed-source companies "leeching" (to use Greg K-H's words) from the kernel devs to benefit.
There's clearly a non-zero (and indeed non-trivial) cost to providing a stable kernel driver API, and the kernel devs don't want to bear that cost, as they feel it would hold back the kernel from evolving, improving on APIs, and fixing security issues or other implementation bugs.
The other side of the coin is that closed driver developers want this stable API to reduce their own development costs, but they will not follow the "intended" workflow that reduces the ongoing maintenance cost to effectively zero.
Unfortunately, that's not very true for FreeBSD. Complex driver modules (e.g., virtualbox, graphics) get broken all the time by kernel API changes.
However you can replace FreeBSD with illumos -- it has a well defined DDI (device driver interface), an OpenSolaris legacy.
> I was NACK to this WSL driver, too, totally flies in the face of the rest of the Linux graphics stack. Companies should get in the habit of talking to the kernel early, and not just dumping a patchset over the fence and hoping we like it.
I guess I'm not sure how much earlier they can bring in the main kernel than a graphics driver that doesn't even display graphics yet. Like I've seen the amount of bikeshedding that happens when talking about hypotheticals with no code to reference.
What should they have done differently in your mind?
The problem is that they built an entirely independent graphics stack. It does not co-exist at all with any of the graphics subsystem which has been built up for many drivers and use-cases over the past 5-10 years.
What they should have done is approached the community with their idea and sought input on how to design their approach. We would have discussed solutions which integrated better into the existing stack. But, then they would be robbed of the chance to do a big announcement and get ahead of the press cycle with their bad driver.
I mean, the were actively asking how to better integrate it before it had gotten to the point of displaying graphics, with some mainline kernel devs questioning if it should be integrated with the graphics stack at all, and be pushed back into the hyperv side of things.
It's very clearly an RFC with an eye for "how can we better collaborate and who should we collaborate with?"
I think they need to better document and support a more opensource stack on the user side of things (maybe a vulkan or mesa targeting the WDDM ioctl interface rather than on the dx12 UAPI), but those are surmountable issues, and makes it look like the amdgpu side of the house.
Full graphics is on the roadmap, basic compute was just an MVP.
Linus has said that he doesn't want discussions without code for the reasons I've stated.
All the complaints I've heard about how WSL didn't do the right thing make it sound like there is no right way, despite passthrough graphics drivers for otherwise totally proprietary stacks on hypervisor stacks from companies with blatant GPL violations ending up mainlined with praise (cough VMWare). And despite the fact that the only way that's been proven to virtualize graphics cards without 2stage MMUs is to replicate the host's ioctl layer into the guest. There's a reason why Virgl isn't used by anyone serious about perf, it's the fundamentally wrong architecture.
CUDA may be an MVP, but it's a damn useful and lucrative MVP.
Also, VFIO mdev can be used to virtualize graphics cards to virtual machines, both nVidia and Intel. Note this is not device assignment (that's "plain" VFIO); it's the host driver exposing a partly pass-through, partly emulated device that virtualizes the host device.
mdev's design was initially a Red Hat-nVidia collaboration with input from Intel (I had a small role from the Red Hat side, mostly coming up with the name :-)) and implemented by nVidia, and is used in production with both Windows and Linux guests.
Not necessarily. Nvidia doesn't have hardware support, rather the guest driver communicates changes to the page tables to the host driver in such a way that the host can establish shadow page tables that merge the guest and host translations.
That's not a great example though, because mdev on Nvidia is proprietary kernel module->proprietary guest kernel module->proprietary user driver.
And in that case too, the ioctl layer is almost entirely maintained across the boundaries like the scheme I'm saying. You need to do that or you almost certainly get a weird impedance mismatch between guest and host that destroys perf.
Even if nVidia's driver has a more lightweight abstraction compared to Intel's, and puts more stuff in userspace, that is not particularly relevant in my opinion: mdev only sits between host and guest driver and by design the host driver only emulates the bare minimum and multiplexes hardware resources instead. What happens in the guest between the driver and userspace does not matter to mdev.
In the specific, I think the relevant part is whether the kernel component and the userspace component are an Aggregate or not for the purposes of GPL virality. From the gnu.org GPL FAQ:
> Where's the line between two separate programs, and one program with two parts? This is a legal question, which ultimately judges will decide. We believe that a proper criterion depends both on the mechanism of communication (exec, pipes, rpc, function calls within a shared address space, etc.) and the semantics of the communication (what kinds of information are interchanged). If the modules are included in the same executable file, they are definitely combined in one program. If modules are designed to run linked together in a shared address space, that almost surely means combining them into one program. By contrast, pipes, sockets and command-line arguments are communication mechanisms normally used between two separate programs. So when they are used for communication, the modules normally are separate programs. But if the semantics of the communication are intimate enough, exchanging complex internal data structures, that too could be a basis to consider the two parts as combined into a larger program.
I agree we're somewhere murky within the last paragraph. The mechanism is separated well enough - the kernel driver is just IOCTLs over a pipe. The semantics are certainly carrying complex internal data structures, however, those data structures aren't part of any GPL program at all.
So perhaps it's not really anti-GPL - but it is certainly useless in a fully Free environment.
It would be nice if Linux kernel devs also had some leverage they could use to increase the amount of open source firmware out there. There is precious little open source firmware and most of it is hardware specific so it becomes less useful as newer generations of hardware come out.
62 comments
[ 3.9 ms ] story [ 123 ms ] threadI’ve seen similar things happen in the CNCF.
It’s understandable but still rubs me the wrong way.
All collaborative projects are inherently political.
Most of the time I see people complaining, it's more from 'I don't understand why I have to abide by the community's values (plural) in order to submit code or otherwise get free labor out of them.'
Well, reputation matters.
Greg Kroah-Hartman let the Habana Labs stuff in over objections, but Habana actually open sourced its libraries later. Qualcomm has demonstrated repeatedly that they will never do that.
And, personally, I'm kinda glad GKH cut Habana some slack. A nice open-source library now exists for that and GKH cutting Habana slack may have contributed to the process.
GKH has also pointed out that he has changed his mind and wouldn't do that again.
People will often do something once because they see it as an exception. Now that this kind of thing is no longer an "exception", people are looking at it differently.
They just keep coming up with new ways to take away the user's control...
Computers as they've been for the last few decades has enabled more people to create so many things than any other time in history. For one of the first times in history, the last 30 years or so has allowed any single person on this planet, with enough money for a computer to do amazing things.
But, this upsets the status quo. Businesses and I guess the 'powers' for lack of a better word, that profited off society as it was have been threatened and in many cases overturned in a short time.
However, they're catching up faster than the average person and through a combination of marketing and just market control, companies are closing hardware platforms faster than what it's taken for most people to begin to utilize all this amazing technology.
And it's sad, I know my kids won't have the experience of booting up an atari computer and laughing as they use their first loop to make a bunch of random things appear on their screen over and over. They won't have that experience cobbling computers together from spare parts and getting on the internet for the first time and just being amazed at all the things out there. They'll never get that feeling of really being free to do anything because they have a computer.
As a kid, even when everything else was really shitty, being able to just turn on a computer and do anything, whether I was supposed to or not, just really helped me through so much and honestly, I don't know how my life would have turned out if I couldn't have done that.
By the time I have kids and they're old enough, unless I buy some retro computers and set it up or something, everything they end up using or seeing will be a locked down device the sole purpose of which is to constantly track them and market to them. The internet is corporate, sanitized, and locked away in these little bubbles provided by platforms.
So much has been lost and slowly but surely, the ability to just pick up a computer and create anything we want is being stripped away by hardware vendors and these huge dominating platforms that control every aspect of the computing experience and feel the need to profit off of and dictate everyone's computer use.
My personal fear is that people will think that they have to explicitly decide to invest into becoming a software developer by buying developer tools the same way you have to invest into dedicated wood working tools to become a carpenter.
The IPad Pro being the worst of the worst. It even markets itself as a "real" computer.
The drivers are probably riddled with security bugs. For the at-home consumer security through diversity and obscurity is sufficient, their adversary is not the government but malware authors. Malware authors can and do use source code as their starting point for exploits. Especially vendored known-bad code, the main thing that is hiding in these drivers.
The manufacturer’s adversary isn’t consumers. It’s not even other small time manufacturers making knockoffs. It’s Google, Microsoft, Amazon and Apple, who are obviously the biggest violators of the spirit, technological and economic, of open source.
As soon as you open source your pathetic network driver, if it happens to benchmark the best in some niche way a Google team needs, it’s not like they go and buy your hardware. It’s just some guy who looks at your source and copies-via-rewriting it internally at Google for the hardware they already own and have source access to. Like how are you going to prove that copyright violation happens? People can barely get GPL judgements against random, low money defendants.
Look at how many businesses they are in. Google is making WiFi routers. Apple makes its own GPUs now. What will they start with? Open source code. It’s not just the AppGet guy, or some niche feature. It’s literally everything, top to bottom. It is the core of the trillion dollar market cap culture.
Nobody has or will ever succeed in a source code copying lawsuit against Google. Apple is going out there and using your shit until they copy it too. Your chips, your wireless stack, your graphics stack, every little thing. They will copy it. Amazon and Microsoft have far flung offices loaded up with people just violating the spirit of open source and legal force of EULAs via copying all day. That is how some guy survives, gets enough work for a year that matters to the bottom line of a multi billion dollar corporation, far out of sight of the compliance culture at HQ.
Every giant company does this. Engineers make careers out of copying open source stuff into some other language, making it seem like it’s clean room implemented when of course it is not. They are the adversaries, not users.
It's a long time since I did it, but I analysed the frequency and severity of identified vulnerabilities, looking at QCOM open and closed components... The results were fairly easy to predict and guess - the closed components had many more critical vulnerabilities, occuring at a far greater rate.
One reason Qualcomm wants to keep these closed is because of IP and the fact they often buy in third party IP (especially in imaging for example). The other reason is because they sell their proprietary userspace drivers to manufacturers as a "board support package", and they need to select the length of time they want updates for. In the old days before treble, when Google changed the HAL every release of android, this meant not being able to run using the old binaries, or having to write shim drivers to wrap old (closed, but exported) function calls with the new expected ones.
Post-Treble, the Swiss cheese nature of their driver code means OEMs need these updates to avoid their phones being compromised, ensuring further recurring revenue.
The best thing that could happen for Android security would be the opening of the entire Qualcomm userspace drivers, so they could be maintained by some competent developers, and eventually unified into an equivalent project akin to the mainline kernel, so that common code could be shared between generations, and therefore security patches can be applied to past devices' components.
And then the next best thing would be for QCOM to open source their modem and RIL (Radio interface layer), because both of those are vulnerability ridden messes of legacy code - I believe their radio baseband firmware is compiled using an utterly ancient compiler still, and lacks even basic secure coding principles. Occasionally you'll see some black box vulnerability research looking into these basebands and they tend to validate the above findings quite nicely.
It's easy to say in hindsight where Android is massively successful, Google will someday allocate resources to recreate Qualcomm, from scratch, avoid patent issues like the consortia did with VC1, and expire the SoC industry. However that seems hard, also Apple has tried this and still hasn't finished an alternative to Qualcomm.
Imagine it was 2006, when there was no one successful to copy. Even if Google were as rich then as it is today, but had the knowledge that Android was definitely going to be a hit, it would definitely not become a phone SoC company - neither did Microsoft nor Blackberry (did Nokia? I honestly don't know). They had the same conviction then in its success, so it didn't matter how much foresight about security issues affecting millions of people there would have been. In any circumstance, they would have not given the kit-and-kaboodle away for free in open source. They'd feel, much like Qualcomm feels today, that they designed something original. Oh, and so now it cannot be shared.
Qualcomm got there first and sells this thing, that's the simplest reason they will never open source. They could achieve all your goals in a difference of execution, it's got nothing to do with the license of their source code. People have written secure closed source software.
For me, there's an important difference between Linus Torvalds copying Unix to make Linux and some giant company copying some other giant company's stuff. Everyone is dancing around the fact that the two are not equivalent and pretending that they are. Someone eventually has to write original drivers that are good, for complex hardware. That person or entity has to get paid somehow. We're being really naive if we just assume that Google, an advertising company that in recent history never starts something unless there is already ample market success to follow (or displace), would do a good job, just because its engineers are highly paid, have lots of time and do algorithms questions well.
>That person or entity has to get paid somehow
In my experience, it is very hard to get companies to even admit what the real cost of open sourcing something like this is. Lots of them just don't even want to talk about it or show their estimates at all, they just say "we will never open source" like Qualcomm does and the conversation goes nowhere. Even if the price is very high and it takes many years, we still need to start somewhere. Then the industry as a whole can chip away at it slowly, because even very slow progress is better than nothing at all. And what's even worse than no progress is if we just give up and sit around waiting for some other company like Google or Apple to foot 100% of the bill to redo all the work over again from scratch.
I believe that people eventually removed the TrustZone checks, but the intent was clear. This isn't Qualcomm directly though, but I imagine it shows part of the root of the challenge here.
https://en.wikipedia.org/wiki/Free_and_open-source_graphics_...
Are there other Qualcomm userspace drivers that need to be freed?
I think there's quite a lot of closed userspace drivers that need (or would benefit from being) freed - I don't have a list to hand, but a quick look inside the /vendor partition (and directory) of a recent phone will give an idea of the closed userspace drivers being shipped on devices.
Qualcomm being closed source is the biggest issue with running good (user-respecting) software on phones, since basically every single Android has a Qualcomm processor/GPU. Samsung has some Exynos stuff but not in the USA. Only because of that, I'm looking into the PinePhone and Librem 5, which can run the mainline kernels.
I realize I'm far from a normal user, but the amount of ill will these companies develop is pretty massive. Broadcom for example has been permanently associated with 'this will be a pain in the ass to get working, and will break your system on updates.' And when I was buying the motherboard / wifi card / wifi router recently I immediately disqualified anything that had a Broadcom chip or didn't specify a chip.
For consumer sales, technologically aware people like us have an amplified impact because people trust us for recommendations. The AMD GPU purchase doubled to $900 because I suggested it to my brother when he needed a new one last month, even though he uses Windows. I've heard this also accredited to the rise of services like Google & GMail; 'techies' used them and their influence on family & friends was a big amplifier to help get users. So I wonder why Qualcomm and Nvidia are so hostile.
Actually there are kernel drivers for several Snapdragon platforms now, to the point that a Poco F1 can boot both GNU and AOSP-based systems off mainline.
Mellanox?
Note that most of the GPU driver problems are on phones and neither RHEL nor Ubuntu officially support phones at all. And there are also many PC components that RHEL/Ubuntu don't support either, although usually that's just because the hardware is too new.
So yes.
Disclaimer, I work for Red Hat.
This is one reason why so many mobile devices run ancient kernels which can't be updated. They stopped maintaining the out-of-tree drivers so people remain permanently stuck on the latest version that still works.
Every release, there's refactoring and improvements of the kernel subsystems. They don't offer stable APIs (by design) for drivers - you need to keep it up to date.
If you want to see this for real, try getting a "code thrown over the wall once" type embedded device kernel tree, figure out the kernel version it shipped on (likely ancient), and try to apply the latest patch level for that release to it. Then resolve the inevitable issues even just applying a few minor bug fixes.
Now if you to to update to the next major kernel version, you'll start to find functions your driver tried to call aren't there, or have new function definitions. You need to dig through the code to see what's changed. Over a few major versions, you'll eventually realise this is a whole lot of bother. If you just mainline your driver, you'll no longer need to worry about this costly maintenance burden - it might well be a few engineers spending most of their time on it. That quickly adds up, and suddenly mainlining starts to make sense.
Nvidia has to keep updating their proprietary driver every release, and presumably has a few developers doing this. I can't speak for how well it works, as I always go for AMD, but other comments here seem to confirm distributions will happily break Nvidia drivers because they aren't mainline and therefore aren't tested.
"Pro" move is to just skip the costly maintenance and keep the kernel same, for old times sake.
Joking aside, I think a lot of this comes down to the incentives (or lack thereof) in the ecosystem - there is no real penalty for releasing an insecure product, or one that can't be kept up to date. Thus inventivising shipping on an outdated kernel you can't maintain, and never patching it.
If those shipping products had to factor in the externalities of this decision, they would rapidly be mainlining everything to reduce the costs of this. The cost of "mainlining" their code will be trivial compared to the (unmeasured) cost of the disposal of the equipment during a reasonable lifespan due to lacking updates, and the replacement of the equipment.
The real problem lies in this practice. Which other popular system uses this approach? Windows, macOS but also open source alternative like FreeBSD all have well-defined kernel API and drivers SDK. So, not providing a standard interface but then crying for the lack of third party drivers is hypocritical: the problem is self-inflicted.
No other popular system follows the same development cycle as the Linux kernel.
There's clearly a non-zero (and indeed non-trivial) cost to providing a stable kernel driver API, and the kernel devs don't want to bear that cost, as they feel it would hold back the kernel from evolving, improving on APIs, and fixing security issues or other implementation bugs.
The other side of the coin is that closed driver developers want this stable API to reduce their own development costs, but they will not follow the "intended" workflow that reduces the ongoing maintenance cost to effectively zero.
I guess I'm not sure how much earlier they can bring in the main kernel than a graphics driver that doesn't even display graphics yet. Like I've seen the amount of bikeshedding that happens when talking about hypotheticals with no code to reference.
What should they have done differently in your mind?
What they should have done is approached the community with their idea and sought input on how to design their approach. We would have discussed solutions which integrated better into the existing stack. But, then they would be robbed of the chance to do a big announcement and get ahead of the press cycle with their bad driver.
It's very clearly an RFC with an eye for "how can we better collaborate and who should we collaborate with?"
I think they need to better document and support a more opensource stack on the user side of things (maybe a vulkan or mesa targeting the WDDM ioctl interface rather than on the dx12 UAPI), but those are surmountable issues, and makes it look like the amdgpu side of the house.
WSL is definitely pulling an Embrace Extend Extinguish. I am not saying the new Microsoft is the same as Ballmer's, but still.
Linus has said that he doesn't want discussions without code for the reasons I've stated.
All the complaints I've heard about how WSL didn't do the right thing make it sound like there is no right way, despite passthrough graphics drivers for otherwise totally proprietary stacks on hypervisor stacks from companies with blatant GPL violations ending up mainlined with praise (cough VMWare). And despite the fact that the only way that's been proven to virtualize graphics cards without 2stage MMUs is to replicate the host's ioctl layer into the guest. There's a reason why Virgl isn't used by anyone serious about perf, it's the fundamentally wrong architecture.
Also, VFIO mdev can be used to virtualize graphics cards to virtual machines, both nVidia and Intel. Note this is not device assignment (that's "plain" VFIO); it's the host driver exposing a partly pass-through, partly emulated device that virtualizes the host device. mdev's design was initially a Red Hat-nVidia collaboration with input from Intel (I had a small role from the Red Hat side, mostly coming up with the name :-)) and implemented by nVidia, and is used in production with both Windows and Linux guests.
And in that case too, the ioctl layer is almost entirely maintained across the boundaries like the scheme I'm saying. You need to do that or you almost certainly get a weird impedance mismatch between guest and host that destroys perf.
And Intel needs something like this either way because their hardware arch is different, and their GPU is heavily integrated with the main IOMMU.
In the specific, I think the relevant part is whether the kernel component and the userspace component are an Aggregate or not for the purposes of GPL virality. From the gnu.org GPL FAQ:
> Where's the line between two separate programs, and one program with two parts? This is a legal question, which ultimately judges will decide. We believe that a proper criterion depends both on the mechanism of communication (exec, pipes, rpc, function calls within a shared address space, etc.) and the semantics of the communication (what kinds of information are interchanged). If the modules are included in the same executable file, they are definitely combined in one program. If modules are designed to run linked together in a shared address space, that almost surely means combining them into one program. By contrast, pipes, sockets and command-line arguments are communication mechanisms normally used between two separate programs. So when they are used for communication, the modules normally are separate programs. But if the semantics of the communication are intimate enough, exchanging complex internal data structures, that too could be a basis to consider the two parts as combined into a larger program.
I agree we're somewhere murky within the last paragraph. The mechanism is separated well enough - the kernel driver is just IOCTLs over a pipe. The semantics are certainly carrying complex internal data structures, however, those data structures aren't part of any GPL program at all.
So perhaps it's not really anti-GPL - but it is certainly useless in a fully Free environment.
https://wiki.debian.org/Firmware/Open