> This also enables third party APIs, such as the popular NVIDIA Cuda compute API, to be hardware accelerated within a WSL environment.
Dollars to donuts this is why Microsoft is implementing this. GPU acceleration is becoming a critical feature for many users (but especially developers) and this will continue. If WSL is to be a serious competitor, this is necessary and I'm glad to see it showing up. This is true of cloud compute, too, and Microsoft is betting big on cloud as its future growth area.
> Only the rendering/compute aspect of the GPU are projected to the virtual machine, no display functionality is exposed.
The Linux gaming folks will be pretty sad about this one. Anyway, this isn't really a Linux port of DirectX. This is GPU compute via DirectX APIs.
So now, I'm just waiting on monitor mode/AF_PACKET for WSL...
Do ML people actually care about DirectX? I thought everyone is using CUDA? Anyway in my university building I have not seen anyone that does machine learning on Windows.
Yes, (almost) everybody doing machine learning is on CUDA. There are multiple pieces of this work, and the DirectX API's are only part of it. Other pieces get CUDA running as well.
And yet another piece is a layer to get OpenGL and OpenCL workloads running on DX12 as well, rather similar in scope to how MoltenVK and the gfx-hal Vulkan Portability work are a layer to get Vulkan workloads running on Metal. This is a big effort, and it seems to me their goal is to get things to the point where stuff Just Works and you don't have to think too hard about the various bits of (technically difficult!) infrastructure to get you there.
They typically don't - the team published both a special build of Tensorflow that uses DirectX, as well as working with nVidia to get CUDA running against their DX Linux kernel implementation
In ML if you decide not to kiss NVIDIA's ass you're screwed. Figuratively they have 100% market share. Having major alternative backend, even if it's proprietary, will force diversity and that has some upsides I think.
The associated blog post (https://devblogs.microsoft.com/directx/directx-heart-linux/) does have more details on exposing display functionality. There is no swapchain functionality yet, but it's clear they're working on it, and part of the work is "DxCore", which seems to be a cleaned up and simplified version of DXGI. So not now, but soon.
Yeah, this should be the right link. TL;DR Microsoft brings DirectX 12, OpenGL, OpenCL, and CUDA to WSL. Vulkan is under investigation. That's really a piece of exciting news!
Yes, machine learning is the first priority as they say in the thread. However, the blog post goes on to say that window system integration is coming. This will eventually be a full graphics stack.
> Anyway, this isn't really a Linux port of DirectX
The entire user mode side of Direct3D is ported, in addition to the user mode parts of the Nvidia, AMD, and Intel graphics drivers.
Please no. Please keep your peanut butter out of my chocolate. Call me a purist, but linux should take nothing from windows, give no ground, make no compromise. One must die for the other to live.
Wait, is chocolate and peanut butter really a thing?! That sounds quite horrible to my non-US ears.
Edit: yep, an online search seems to say that's an actual thing. I guess I'm part of the ten thousand today https://xkcd.com/1053/. I will never understand the US fascination for peanut butter.
> I will never understand the US fascination for peanut butter.
At one point in history, US farmers were encouraged to grow peanuts as a rotation crop to improve soil quality. That led to a glut of peanuts in the market, so people tried to find uses for them. Peanut butter was invented+ as one of these uses, and has been a staple of American diets ever since.
And yes, as a sibling notes, Reese's peanut butter cups are actually alarmingly tasty, but.... as with any $1 chocolate bar, that's shitty HFCS-saturated chocolate and shitty palm-oil-laced peanut butter, with way too much sugar in it, so if you're too good for that, well, that's a credit to your tastebuds, good on ya.
So eat real chocolate with real peanut butter. Real peanut butter is nothing but peanuts and salt (it keeps well, but fresh-ground is better). Real chocolate, I trust you can figure out. Milk and dark are both good in this application.
Although, of course, peanuts are not true nuts (no more than macadamia or almond or walnut), they're nonetheless very nutty, and the effect is pretty similar to "almond bark", or hazelnuts with chocolate, or pecans and chocolate. And of course you can just eat peanuts with chocolate, an okay combination. But there's something weirdly perfect about peanut butter with chocolate, better than peanuts with chocolate.
But hey, although I'm not American, I am from America's hat, and I do like peanut butter in a few other formats too.
My guess is that MS and Apple are both slowly trying to steer their ship in the same general direction as ChromeOS: a stable, locked down OS that runs applications in dedicated sandboxes/containers/VMs. No longer does the OS need to provide the same "shell" to those applications. You don't need a library-based wrapper to the syscall layer. The paravirtualized hardware is the new syscall layer. You can wrap whatever OS interface you want around that in order to support different kinds of workloads. Games can run as close to the system as possible. Workloads destined for the cloud can run in a Linux environment. Instead of being intermediated by clunky VM kit from third-party vendors, they'll provide a lot of it themselves to optimize performance and ensure adequate security between environments in a user-friendly manner.
By making the virtualized hardware the "glue", they can avoid the GPL/copyleft infection of their commercial OS, while supporting different kinds of developer experiences.
That's more a matter of buffer management. Though it may turn out , if their work is too tied to NVIDIA, to only support NVIDIA's de facto proprietary selection: EGLStreams.
Though in principle, there's nothing preventing them from using GBM instead of EGLStreams, and there are some good practical reasons such as having compatibility with the broad base of existing accelerated Wayland windowing libraries and applications.
You can use Wayland with NVIDIA drivers. The problem is that NVIDIA and the open source drivers expose different buffer management APIs and Wayland does not abstract over that, so it has to be explicitly handled by every client application. Some Wayland clients refuse to support both, others had NVIDIA support patched in by NVIDIA itself.
AMD started to support the development of the open source driver several years ago by publishing hardware specs. . I am not sure if they switched completely or are still maintaining a closed source driver on the side - I haven't bought AMD cards in years, my last one only "works" with the binary blob.
The people working on the NVIDIA open source driver have no official support and were fighting with signed firmware blobs last I heard. I wish them luck, but even on older cards it is more likely to crash your system than render anything.
Single data point here but I just switched from an NVIDIA GTX 960 (2015 card) to an AMD RX 570 (2017), both using open source drivers, and the performance improvement in Wayland/Sway is huge.
My understanding is that even 2015 is too new for nouveau to run with high performance due to something called reclocking, where the card starts up at a minimal clock rate and then it's up to the drivers to reconfigure it for running at the advertised clock.
> These changes are on the WSL’s team roadmap and you can expect to hear more about this work by holiday 2020.
As if describing things in terms of northern hemisphere temperate seasons wasn’t bad enough (and still worse commonly showing how little you care about any place other than the USA and maybe Canada by using the name “fall”), now we have this: “holiday 2020”. I don’t know when this is talking about. I’d have guessed northern hemisphere summer school break first, but I guess that’s just about finished now, so it can’t be that. Christmas time would have been my next guess, but surely you’d describe that as “by the end of 2020”? And then other possibilities occurred to me—Halloween? Thanksgiving? I have no idea at all what Americans would call “holiday” as a time of year.
Holiday == November or December. Thanksgiving and Christmas are the big ones, but we use generic terms like "holiday" to accommodate people who celebrate different holidays around those times. This is pretty common terminology; IIRC the "War on Christmas" has been a thing for a couple decades.
> I’d have guessed northern hemisphere summer school break first, but I guess that’s just about finished now
Summer break more or less just started for most students here. The fall term will start around August.
That would be great news - something I asked for since the presentation of WSL - I just hope it uses the new graphics driver infrastructure, as the first implementation mentioned in your link seems to be using RDP, which is less efficient.
But if it does, that would be excellent news.
Yup, from one of the MS staff replies further in the thread[1]
> There is a single usecase for this: WSL2 developer who wants to run
machine learning on his GPU. The developer is working on his laptop,
which is running Windows and that laptop has a single GPU that Windows
is using.
Can't say I can get behind MS trying to shift maintenance for a Windows only "feature" onto the Linux devs here.
That's pretty much always a possibility. On the other hand, they're also generally fairly open to add things as long as the developers react to concerns. I'd guess if this can be neatly stuffed in a corner and treated like any of the other Hyper-V specific drivers, and quality is okay, it has a reasonable chance to be accepted.
And if it is rejected, Microsoft can still ship it in the kernels for the distros they offer on WSL.
Yeah not the end of the world if it doesn't land immediately, based on the original devs responding that they'd rather find the right place for it than the expedient place for it
If you read the replies by Dave Airlie and Daniel Vetter, it seems somewhat likely that upstream won't accept it. Perhaps that's just initial skepticism that will evaporate after more discussion, but perhaps not.
Frankly this does just seem like MS wanting to reduce their maintenance burden on what they expect will be a very important part of their WSL offering on Windows. There's nothing inherently wrong with that desire, but the people on the other side need to weigh their maintenance burden[0] with what benefit this will have to the Linux community as a whole, which at first blush seems minimal. Especially considering that the userland pieces that talk to this driver aren't open-source.
There's also the question of whether or not you believe WSL as a whole is good or bad for Linux. If there are people who would run a Linux desktop for development who then decide not to because WSL exists, perhaps that's a bad outcome. If you have people writing more DirectX GPGPU code who would otherwise write to a standard interface like OpenCL, perhaps that's a bad outcome (to be fair, there's also a lot of CUDA out there, which is similarly problematic). Is this the start of MS going back to their "Embrace, Extend, Extinguish" playbook, or is that just a paranoid fear? They've definitely been embracing Linux, and enabling people to write DX12 GPGPU code that targets a Linux environment but will only run under WSL on a Windows install does feel like "extend".
I'm not sure where I personally stand on this issue as I haven't done my research, but I think they're interesting questions to ask.
If this gets rejected, of course it doesn't stop MS from doing any of these things, but it does make it harder for them to maintain their extensions to Linux.
[0] Airlie is even concerned that just by looking at the code, he or other DRI developers could run into future IP derived-works trouble when designing future Linux graphics interfaces.
> There's also the question of whether or not you believe WSL as a whole is good or bad for Linux. If there are people who would run a Linux desktop for development who then decide not to because WSL exists, perhaps that's a bad outcome.
I think if that really matters you should be against anything in the kernel to make it work well as a VM under Windows or MacOS or BSD, and in VMware or VirtualBox too.
From that point of view, Linux in a VM on another host is taking away from Linux running as a desktop on that host. Linux running as a VM on a VMware cluster is taking away from Linux running on those bare metal servers instead of VMware ESXi.
I think the more sane way to look at it is that Linux is an application (and its subcategory is that it is an OS) which is meant to run on various hardware and software platforms, the more the better. This strategy has worked very well over that last couple decades.
Does allowing WSL mean that some people that would install Linux on their hardware just run Windows instead and use Linux on top? Probably. Does it mean that people that already ran Windows and have never used Linux get exposed to it for the first time and get familiar with it through a few click on their existing Windows computer? That's also probable. Does it really matter in the end? Probably not.
> I think if that really matters you should be against anything in the kernel to make it work well as a VM under Windows or MacOS or BSD, and in VMware or VirtualBox too.
Why? I would guess that a very large share of linux kernels run under a hypervisor in some data center, in a public cloud or some OpenStack cluster. Won't those mostly be the same features?
Because WSL doesn't compete with people who would otherwise run a Linux box, it competes with people who would otherwise shove Ubuntu into a VirtualBox and run it on their Windows box in seamless mode. So the idea that it drives people away from Linux proper is nonsense (BTW., WSL is Linux proper), unless you also believe that installing a Linux in a VM on a proprietary system is also driving people away.
> ... shove Ubuntu into a VirtualBox and run it on their Windows box in seamless mode
Given that the WSL2 rewrite is essentially this, without even the niceties of a VirtualBox GUI wrapper to control the settings, I keep wondering what all the fuss is about.
>>>There's also the question of whether or not you believe WSL as a whole is good or bad for Linux.
IMO it's a good thing. Given that windows accounts for 90%+ of the desktop OS share, Windows might very well become the world's most used Linux distro.
It is, of course, decidedly not a Linux distro though. If it was, it wouldn't be an issue. I think there are positives, but it looks a lot like "Extend" to me.
I can't even say I wouldn't use it - it might be nice! But I will not use any WSL-only capability, that's for sure.
My understanding is that this is meant to become transparent to the user, that really this is about enablement of hardware acceleration within WSL, and that the typical Linux userland graphics APIs like OpenGL would layer on top. So the goal isn't to get you to link to libdx12 or whatever it is they have here, it's actually a piece that will be used by Mesa to provide accelerated GL for plain old Linux apps to use when running in WSL. It seems like the easiest bite off the apple was offscreen rendering and GPGPU functionality, but the MS devs seem willing to work with the kernel devs to rearch it so it fits into the typical Linux graphics stack ie DRI and other lower level systems. As far as I understand, this would be required for, say, Wayland to be able to have hardware acceleration when running in WSL.
I'm still piecing it all together, and I definitely feel that "Extend" feeling, but I'm not sure that's what's happening here. Looks more like a few devs at MS are trying to solve the GPU Accel use case for WSL...
Hi. Microsoft PM working on WSL, Terminal and Windows.
WSL2 literally runs user-mode distros (and their binaries) in containers atop a shared Linux kernel image (https://github.com/microsoft/WSL2-Linux-Kernel) inside a lightweight VM that can boot an image from cold in < 2s and which aggressively releases resources back to the host when freed.
So when you run a binary/distro on WSL2, you are LITERALLY running on Linux in a VM alongside all your favorite Windows apps and tools.
If some of the tools you run within WSL can take advantage of the machine's available GPUs etc. and integrate well with the Windows desktop & tools, then you benefit. As do the many Windows users who want/need to run Linux apps & tools but cannot dual-boot and/or who can't switch to Linux full-time.
This will (and already has) resulted in MANY Windows users getting access to Linux for the first time, or first time in a while, and are now enjoying the best of both worlds.
The question isn't asking whether you, a Windows user who runs Windows, benefit. The question is asking what it does to Linux users who don't run Windows even a little. (And I think you know that.)
That's like asking whether Linux users who don't run ESXi benefit from their paravirtualized drivers being upstreamed. No, they don't, but they were accepted with way less bruhaha. And that's despite VMWare blatantly violating the GPL for more than a decade.
With DirectX on WSL, you can do new things when Linux is running on Windows (via WSL). But these new things aren't possible when Linux is running another way (e.g. on the bare metal).
So people who use it are married to Windows.
I think folks would be absolutely excited if this was an initiative to allow writing DirectX applications on Linux, and available for Linux on bare metal. But as people realize this marries them to Windows, they go meh.
I think the concern with this DirectX implementation is that it only works for WSL users, not standard Linux users. So, it's a software API that will only work in your ecosystem, not the overall Linux ecosystem.
If DirectX on Linux could also work on bare metal, the conversation here would likely be different.
> [0] Airlie is even concerned that just by looking at the code, he or other DRI developers could run into future IP derived-works trouble when designing future Linux graphics interfaces.
And that is, IMHO, a very real concern and it really should not be merged.
Yeah, I think MS should address this point. Airlie also asked if everything here is covered by Open Invention Network (OIN) of which Microsoft is a participant, and indeed I'm pretty sure it all has to be since that is the point of OIN. But the MS devs will now need to have legal sign off that it's all covered by OIN before they can respond in the affirmative. I assume that will take some time as these things usually do. Especially since this seems more like a bottom-up effort than something top-down managed
Why would it? That's a functionality that can't be used without Microsoft proprietary parts. Unless I missed something, I don't think there is an open source implementation of DirectX somewhere?
I doubt the linux devs ever see WLS as a target they have to maintain themselves.
Ok, not my field, but isn't it based on OpenGL? Does it count as an implementation or an emulation layer? My understanding is that Microsoft is trying to give WSL a transparent access to the GPU using the regular linux interface and transmitting it to DirectX.
Using that with wine would mean adding two emulation layers before reaching the actual driver. I fail to see any use case for that.
It's based on vulkan, which means that it has low enough level access to the GPU that I think it's fair to call libraries implemented on top of it "native". Most GPU drivers have some kind of translation for directx already, and there's no inherent reason why the open source directx implementation has to perform worse than the GPU implementation of directx. I hear that the open source dx11 implementation actually beats AMD's DX11 implementation in some cases.
So yeah, vulkan is neat and opens up a whole lot for the linux world. In the future you'll probably see userspace implementations of opengl on top of vulkan, maybe even CUDA implemented on AMD gpus, although I'm not sure how practical that is. Also a whole lot of exciting GPU sharing tech, accessing GPUs inside of VMs for example.
DirectX will come to linux, but it won't be thanks to microsoft. You can thank valve hedging their bets on the microsoft store for that.
What would be the big loss other than MS bundling a ppa ? Nvidia drivers and cuda anyway need a ppa. So wsl + nvidia + cuda being a single ppa is anyway not that far off.
To be honest I don't quite understand what stopped those developers from running machine learning on their GPUs under Windows itself. Most frameworks work just fine. I've been doing quite a lot of TensorFlow with both Python and .NET.
The only time I faced the need for Linux box is trying a demo project from OpenAI, which did not use the features it required Linux for on a single machine anyway.
I believe the purpose is for devs who are deploying to Linux, their toolchain may not fully work in Windows, but they want to have a similar dev/test env within Windows... pretty much the whole point of Windows Subsystem for Linux (WSL). It's not that the frameworks don't work on Windows, it's the deployment tech, like Docker, Ansible, their build scripts etc etc. Some users may be doing something totally custom on top of the GPU without a framework layer but that is probably very few users. I'd have to guess though that those users would be the type of users that would get MS to go through the effort though...
Do people actually use Docker and Ansible a lot for ML???
Doing something custom on top of the GPU is also not much different on Windows, than Linux. CUDA is basically the same. OpenCL and Vulkan are available too.
I'd like to hear perspective of a person, who actually does ML specifically on Linux for some reason.
Well CUDA is only one thing. Especially in the python environment it's only a matter for time before you run in some wierd dependency issue that is Windows only. For example, getting XGBoost completely up and running on Windows requires you to either build it yourself or download a .dll from a university link. Installing it on Linux is just a proper pip install.
Also windows not having a build in C compiler makes you dependent on the horribly convoluted Visual Basic stack that seems to have a lot of dependencies for some python ML libraries. Docker makes it a lot better to run and I almost always deploy in a docker container because the ML modules I deliver are often interacted with as a black box with a REST API on top.
It looks like Anaconda supports XGBoost on Windows.
You might be right about C compiler. But something itched when you mentioned Docker. Could getting Windows SDK installed be harder, than installing Docker?
I just use chocolatey for that
> choco install windows-sdk-10.0
That being said I had to avoid installing VS 2019 for quite awhile because Node.js native module build chain couldn't work with it. There are complexities
I do all of my development on linux, if I can, but to be honest the GPU support is generally better on windows because that seems to be the main platform AMD and NVIDIA target - though linux support is not too bad. GPU support is the only potential benefit in using windows that I can think of though. Everything from package management, to build tools, FOSS support, community, troubleshooting, etc. is generally better on linux.
Things like filename limits and command length limits seem to get hit on windows way more frequently, making all code more fragile in general, and a million other little things.
Even popular libraries like zeromq don't support namedpipes on windows because of how complicated they are and how different to everywhere else.
Just determining what visual studio version is installed seems to trip up projects all the time.
Mostly doing several kinds of NLP:
My actual setup is a Windows Laptop to SSH into Linux machines w/ tmux session. However, I really appreciate WSL for working offline, etc.
My main reason: It is the most convenient way to have Unix tools (grep/sort/cut/sed/less/...) and bash available. Cygwin always was a pain, MinGW / GitBash felt much better, but ultimately WSL just feels best.
These tools are incredibly valuable to my workflow. Sure, stuff like pandas can be nice for small datasets, and some data sits in some DB/Kafka/distributed system. But there have been countless cases where unix tools allowed me to take xxGB zpfiles of text and do basic examination or even build baseline models within a few hours.
Sure, there always are alternatives to use these tools and there are many equivalents. But I would always prefer WSL + conda for Linux to a typical "Windows Conda" installation with that weird GUI and the need to install so many different applications to even just look into the first or last few lines of a huge textfile.
EDIT: That said, of course I can/could always just run a juypter notebook under windows using windows cuda + GPU and share files with a WSL bash where I do my modifications. But again, everything within the same systems just feels better (ipython shell magic, no worries about if paths to the same file are really identical, etc) and while this is by no means a game-changer, it is just nicer that way.
The reason to use Linux is to follow the crowd. I use Nvidia's docker container because the plurality of devs use it. Over the course of my career I've found that well-trodden paths tend to have a _lot_ fewer bugs thanks to other people finding them first, and when I do get some
I don't have to spend time explaining or justifying or isolating my setup.
Conversely though, my work is itself off the beaten path enough that I'm likely to run into weird bugs. If I was pushing images through a CNN, that'd be well-trodden enough on every platform that I'd be a lot less fussed about which particular platform I use.
This mainly to woo developers who are now working on Mac OS or thinking to use Mac OS to use Windows and WSL combination instead for ML & AI application development.
My experience with GPU accelerated development is quite horrible on Windows for anything other than the NVIDIA prepped docker container. There was always something missing or some drover was incompatible. In the long run I have always regretted developping Python on Windows, also often because whatever was developped was to be deployed on a Linux box.
I do not think it's purely windows to blame here though. It's only quite recently that NVIDIA started fixing their documentation and instructions on getting all the right CUDA CuDNN stuff running properly on a system.
Imagine if you could run AI/ML apps and tools that are coded to take advantage of DirectML on Windows and/or atop DirectML via WSL.
Now you can run the tools you want and need in whichever environment you like ... on any (capable) GPU you like: You don't have to buy a particular vendor's GPU to run your code.
If you're old like me and remember the dark ol' days when games shipped with specific drivers for (early) GPU cards/chips, but failed to run at all if you didn't have one of the supported cards, you'll understand why this is a big deal.
> If you're old like me and remember the dark ol' days when games shipped with specific drivers for (early) GPU cards/chips, but failed to run at all if you didn't have one of the supported cards, you'll understand why this is a big deal.
Maybe I'm not that old, but I'm old enough to remember the days when microsoft was intentionally degrading opengl performance on windows ;).
This. Some games would have a handful of different renderers for different setups, while other games would only support one specific card type (and if you were lucky, a software renderer).
Those days sucked. Bigtime. If we can avoid doing the same mistakes for machine learning then we should.
> Maybe I'm not that old, but I'm old enough to remember the days when microsoft was intentionally degrading opengl performance on windows ;).
Which is still nonsense, since this only affected the OGL driver shipped by Microsoft. In contrast to truly bad actors like Apple, OEM were free to ship their own OGL drivers from day 1.
Don't you think the effort to achieve this would be absolutely massive? I don't know what kind of resources are thrown on this project, but I'd estimate minimum to be 3 dev teams for 2 years just to get a few variations of ResNet to work "as is". And that's just for regular models, that don't require quantization or (auto-)mixed precision for training.
>Now you can run the tools you want and need in whichever environment you like
Isn't the linked post saying you have to be running on Windows though? It seems like it would make way more sense to either port directX to Linux, or ditch directX and put those resources into supporting Vulcan.
Neither pytorch nor tensorflow support WinML, so this is going to be a bit of a stretch still, since CUDA is still the toolkit of choice for mainstream ML frameworks.
> horrible on Windows for anything other than the NVIDIA prepped docker container
o-O nvidia-docker does not even support Windows.
I think the only thing you need to know is which CUDA version your cuDNN requires, and it was quite clearly stated on the download page. Also the same on Linux. For nvidia-docker you used to need a specific driver version.
Lots of ML frameworks are built for Linux/UNIX first.
The OpenAI projects you mentioned are a good example, but also projects like Ray (ray.io). Even PyTorch - a lot of stuff works fine, but their parallel DataLoaders actually work slower on Windows than the single-threaded ones (see this github issue: https://github.com/pytorch/pytorch/issues/12831 )
But Windows is a painful OS to use for anything other than gaming. Ideally, I'd like to see the exact opposite of this: run Linux with a Windows subsystem just for gaming.
What do you mean? Wine has a different architecture because distributing microsoft binaries is not legal, so technically it's not the same thing, but it still does an amazing job and a lot of apps/games works flawlessly.
It does an amazing job, but a lot of apps and games do not work flawlessly. Or didn't, last time I tried Wine. Maybe this situation has changed a lot since then, which would be awesome.
Holding software to that standard eliminates most of it. "No, I haven't tried Google Docs yet. They're still adding features and fixing bugs. I'm holding out until it's stable."
Context matters a lot. I'm not holding Wine to unreasonable standards, its a matter of recognizing reality of the situation and that Wine is not such a exact mirror of WSL and as such it will continue to have significant issues.
And a lot don't. And then you have to spend a lot of time researching why and messing around with config options and maybe even compilers. And the games sometimes stop working after an update.
Conversely, if you run Windows, it's rare that you need to work hard to run a game.
This is definitely not my experience.
Off my 100 games on steam, gog and egs, 2 doesn't work straight out of the box.
- rocksmith 2014 that can work by switching to alsa audio but the audio lag make the experience subpar
- bit trip beat that i know can work by changing something but didn't try
Apart from photoshop video production audio and many other professional uses.
Like the previous response what does this buy me as compared to developing and running natively in windows - as there are native compliers that support cuda etal on windows.
Wine / Steam's Proton does a decent job, and some older games even work better with Wine than with Windows 10.
If you have a spare GPU, a VM with PCI passthrough does an even better job, except for some anti-cheat software that artificially discriminates against this setup.
In theory it ought to be possible to switch a single GPU to/from a VM without a reboot. In practice I have no idea how huge a refactoring to the Linux graphics stack that'd require.
> "some older games even work better with Wine than with Windows 10."
This has been true for 16 bit games since long before Windows 10. Ages ago one of my favourite games stopped working on Windows, but Wine had no problems with it. So my impression has always been that Wine is excellent for really old games, but slightly more recent games, it could already be very hit and miss.
> "If you have a spare GPU, a VM with PCI passthrough does an even better job, except for some anti-cheat software that artificially discriminates against this setup."
Doesn't every CPU these days have onboard graphics? My Thinkpad X1E should support hybrid graphics, so it'd be nice if I could give the GPU to a VM and have the desktop use the CPU graphics.
But if a Windows VM does a better job, that means Wine doesn't yet do as good a job as Windows. Though it's certainly true that Steam support for Linux is growing. But I don't think every Steam game already works on Linux.
Wine has gotten very good at recent games... or more specifically, Proton has. (Proton is a Steam-maintained fork of Wine, and is built in to the Steam client.)
Official Proton "support" is limited, because it requires certification by Valve and/or the game developers that the game runs well (the equivalent of a "native" rating on winedb/protondb), but if you're willing to go down to "gold" levels of support it still runs 70-80% of all Steam windows games.
> Doesn't every CPU these days have onboard graphics?
Don't bother attempting GPU passthrough on any laptop with an AMD CPU (eg Ryzen 2700U) and Radeon GPU (eg RX 560X).
It turns out the GPU passthrough needs a dump of the Radeon BIOS provided as a file, but no-one can dump the BIOS of discrete AMD laptop GPUs. :( :( :(
> In theory it ought to be possible to switch a single GPU to/from a VM without a reboot. In practice I have no idea how huge a refactoring to the Linux graphics stack that'd require.
I've got this working today. I do it through swapping the nvidia driver for the vfio-pci driver (and back again if required). The slight annoyance is that you may need to restart X11 (for me this is not an issue).
Windows is a painful OS to use for anything other than gaming and web browsing, I agree. (Well, it was, until WSL.) If I ever again own a work-first non-gaming desktop, I will once again install some Debian GNU/Linux on it.
(My gaming desktop, of course, runs Windows.)
On laptops, however, I continue to find myself jumping through idiotic hoops to use Linux. The driver support is always just-barely-good-enough. Maybe it's audio, maybe it's graphics, maybe it's power management. Maybe it's something involving networking-after-power-management or some crap involving "don't unplug your headphones while the lid is down". On my current work laptop, a beautiful Thinkpad Carbon X1, I can't get the power management to work properly, so I just have to accept that there's no hibernate. I'm constantly forgetting to shut down, put it in my backpack, and then pull it out drained. What a pain in the ass. Could someone fix this problem, probably. Can I? Not in the dozens of hours I've put into it. I hate doing IT, I hate it I hate it I hate it.
However much Macs make me want to vomit in my mouth, I can see the appeal. The drivers work at least 90% as well as Windows drivers, and the UX is at least half as good as a lightly-tuned Linux machine. "Jack of all trades, master of none, is oftentimes better than master of one"
Anyway, before this lockdown ends, I'm upgrading my laptop distro to this new distro I've heard of out of Redmond, I think it's called "Windows".
Many companies, including the one I work for, have most development happening on Windows. It is far easier to manage for Enterprise use-cases, with all relevant services integrated and obtained from a single vendor.
Not to mention that there are quite a few development environments for more obscure platforms that still only exist for Windows.
Overall, since most development time is spent in an IDE, the OS is really of little relevance to software development. Sure, some people insist in using command line tools, and that is unlikely to be pleasant on Windows, but a lot of other developers don't, and we couldn't care whether we're running our Emacs on Linux or Windows or Genera or whatever.
I disagree, Windows is very painful for me to use at a basic level compared to Linux. I would be very unlikely to take a job that forced me to develop under Windows.
The same is mostly true for me when trying to use a Linux desktop.
However, if I'm using IntelliJ or Emacs and Firefox, I don't really need to care what OS is running underneath too much.
Edit: of course, Linux and Mac are available for devs that prefer them. It's still much easier for IT to manage 7000 Windows desktops and a couple hundred Linux ones than it would be to manage 7000 Linux desktops.
I'm not sure I agree with that. 7k "normal" machines for any given value of "normal" seems like it would always be easier than 7k "normal" machines and 200 oddballs. Is the sysadmin tooling for Windows really that much better?
I don't understand what "being managed entirely hands-off" means. I can assure you there is no manual intervention from IT on every desktop system in the fleet, so I would call that exactly "being managed hands-off". I am certain there are always a few systems that do need some manual intervention for whatever reason, but that seems to just be the way with computers. It's not like Ubuntu updates never break, or no one ever gets to install a broken package before it is retracted.
I mean with 'entirely hands-off' that the computer just keep on running. On Linux, most packages are part of the repository and update automatically. On Windows, those packages have to be updated manually. Even if everything is automated, the computer will have to be restarted quite regularly. At least to me, Windows seems to be more difficult, especially if you have a fleet of desktop systems that can be chosen to be perfectly Linux compatible.
Where does the complexity on Linux come from that makes managing them more difficult?
Even on Linux, you need restarts if you want the updates to shared libraries to actually happen. You can't just apply a critical security update to OpenSSL for example and hope that the user actually restarts their programs at some point - if you care about the update being applied, you need to know that by some date all of the programs on that system are actually restarted, and a scheduled reboot is by far the simplest way to do this.
Then there's the question of pushing an update to all managed computers. Maybe it's not a package update, but you want to change some SELinux policy for all users, or update some DNS server or the default search domain and so on.
Never mind the question of how you can instruct one of those Linux computers to delete all data it holds whenever it next connects to the internet (to handle the case of a stolen company laptop).
There are so many things that you need in an enterprise setting that have common (though probably quite expensive) tools available for Windows. Maybe some of these exist for Linux as well (I would expect RedHat to have some), but I'm not sure. Linux admin is usually reserved for servers much more than desktop computers.
Why would you want to delete the data? That's when you open the door for an attacker to access it. If the data is encrypted, it can remain on its partition because it is inaccessible.
Interestingly, apples have to be compared to oranges. On Linux, it is easy to identify the programs that are using a library. Thus it is easy to restart just the services that are patched. In general, things can be scripted so there are no tools available. But this requires somebody who understands the system. From a business perspective, this might be more expensive, or not, if the tools are expensive.
I’ve programmed in a professional capacity on macos, windows, and linux. Windows used to be the worst, but with WSL and VSCode, i would say it has overtaken macos in my mind, because WSL is better than using brew
Sure, companies may love it, but developers hate it.
I use a Windows laptop at the company I currently work for, because everything is locked down and I wouldn't be able to get my own laptop connected to the network. (Or so I thought; I co-worker managed to use the Windows laptop as a bridge to his own Macbook.)
Now you're right that as long as I stay in the IDE, it's not so bad. But every once in a while I need to do something outside the IDE, and I immediately get slapped in the face by how stupid some things are. And because it's an enterprise environment, some things are even worse than usual; opening a folder, or saving something, can be unreasonably slow because either it's a network drive or it needs to be checked for viruses and malware while I'm trying to use it. Or for some other reason. I don' t know, I just experience the extreme slowness.
Also, on top of the old terrible DOS shell, there's now also a Power Shell that's supposed to be better. It apparently has some powerful features I don't really grasp, but it's still not remotely as good as bash. And sometimes the command line really is unavoidable.
But the real pain is at home. When I activate Windows 10 on a new machine, I need to create a Microsoft account. I don't want one, but it takes serious determination to avoid it, because behind every message is another trick to sucker you into an MS account. When you finally do manage to create a local account, you're immediately expected to compromise your security with 3 insecurity questions, and no way to avoid it as far as I can tell. Previous versions of Windows did not have this stupidity.
Also, somehow Windows keeps losing my mic, speakers or camera. Once I've found the right troubleshooter, it immediately figures out how to fix it, which is great, but it also keeps losing them again. And finding the right troubleshooter takes a couple of steps and a bit of searching. I feel like I need to pin several relevant troubleshooters to the taskbar.
And then there's the total lack of access control. To install anything, you need to be admin. I gave my son a restricted account, but he can't do anything with it. I'd like to be able to create an account that can instal games, but can't compromise the system. No such option in Windows. If you can do anything, you can do everything. Unless you're in an enterprise environment, in which case you often still can't do anything. So I guess more detailed access control does exist, but only for enterprise users or something.
You generally have some good points, and some points that have more to do with preferences, or luck with hardware. I would note that the lack of anti-virus software is one of the reasons a lot of companies don't want to run Linux on employee systems. Locked-in Windows systems at least have the advantage that even if you Run as Administrator some .exe that you got emailed, there is a good chance that your AV will not allow it to run; if you're running something as sudo on Linux...
> And then there's the total lack of access control. To install anything, you need to be admin. I gave my son a restricted account, but he can't do anything with it. I'd like to be able to create an account that can instal games, but can't compromise the system. No such option in Windows. If you can do anything, you can do everything. Unless you're in an enterprise environment, in which case you often still can't do anything. So I guess more detailed access control does exist, but only for enterprise users or something.
Here I never understand this point. You can't do anything on a Linux system if you don't have sudo access - it's not like apt or yum have any special magic to allow non-admin users to install stuff. And if you can install software on a system, you can already do anything else. Especially Games, which install drivers to perform DRM and anti-cheat bull.
Now, if you want to look into it and waste quite a bit of time, Windows does allow you to configure access control at a very fine-grained level for access to non-system folders. But as long as the installers want to install things in system folders, there really isn't any solution.
I admit I've never really looked into how detailed Linux is in access rights, but on Unix systems, it's very normal to have install rights for specific directories. If Linux doesn't allow that, that would be disappointing, but I strongly suspect Linux allows this just as much as other unixen. So that would mean you can install stuff without sudo rights as long as you get group rights to the right directory. And that's a much safer approach to security than all-or-nothing.
I'm pretty sure neither `apt` or `yum` or other common package managers support any way of running as non-root. Of course you can download the sources and compile yourself, or maybe even find a binary distribution with all dependencies included (good luck with that).
Painful? What exactly is painful? Apart from the Settings/Control Panel debacle, I don't know any issues you could be facing on Windows, unless you do a lot of C/C++ development which is still problematic due to the lack of a proper package manager.
I must disagree with the C/C++ take. Visual Studio is still, for me, one of the best IDEs out there, and the single best one for C/C++ development. And for the longest time, Windows indeed didn't have a good package manager, but over the past few years we've had vcpkg, which fills the vacuum pretty well when it comes to getting libraries without much hassle.
If your dev or production stack is Linux based, I think it makes sense to try to bring the GPU to your dev stack instead of the other way around. If you're working with other devs who are on actual Linux stacks, it'd be a pain in the ass to always require for there to be hybrid Windows/Linux tooling.
> Can't say I can get behind MS trying to shift maintenance for a Windows only "feature" onto the Linux devs here.
Interesting take on the situation. This is effectively a driver they need to get into the kernel (just one that targets a paravirtualization host and not “real” hardware), and Linus has been adamant that the correct way to write a driver in Linux is to upstream it into the kernel.
The perspective that upstreaming a driver into the Linux kernel is a burden for Linux kernel developers is one I haven’t heard before, and seems to clash with Linus’s typical stance. Is this something that has some prior examples? Genuinely curious.
Just run a Linux hyper-v vm. That's what WSL2 is doing under the hood anyway. I run it this way and it's great. I have windows terminal auto ssh into it. Performance is great. And using the X server x410 on the windows side gui performance is fantastic (though no hardware acceleration) because instead of ssh tunneling x410 suports AF_VSOCK for the x socket, which hyper-v supports for performance as good as a domain socket on the same machine.
I've had trouble researching if WSL2 is in fact a hyper-v managed VM. I've seen some documentation referring to WSL2 as a tightly integrated Krypton (scaled down hyper-V) VM. It seems to imply the host overhead isn't as high as a guest on hyper-V
AFAICT Krypton is stripped down in the sense that a lot of the management framework is gone, but as far as the guest is concerned, it's running on hyper-v.
WSL uses a Hyper-V derived virtual machine that is
* Sparse & light - they only allocate resources from the host when needed, and release them back to the host when freed
* Fast - it can boot a WSL distro from cold in < 2s
* Transitional - these lightweight VMs are designed to run for up to days-weeks at a time
Full Hyper-V VMs aim to (generally) grab all the resources they can and keep hold of those resources as long as possible in case they're needed. Full VMs are designed to run for months-years at a time.
WSL's VMs are MUCH less impactful on the host - FWIW, I run 2-3 WSL distros at a time on my 4 year old 16GB Surface Pro 4 and don't even notice that they're running.
"There is currently no presentation integration with WSL as WSL is a console only experience today. The D3D12 API can be used for offscreen rendering and compute, but there is no swapchain support to copy pixels directly to the screen (yet )."
This leads me to believe that display support is intended in the future. It's a work in progress. They've gone this far why would they stop at compute? Still, it's pretty awesome if you ask me.
They've announced that Linux GUI apps are coming later this year to WSL2; while that is possible without GPU, I imagine MS wants a decent UX for the feature, though, which suggests...
Because older games run better on wine than they do on windows. Though there's more recent software available to mitigate some of that (wined3d, winevdm, etc).
And wouldn't be faster simple use native Linux installed on the computer ? Instead of running on a VM, and a glue/translation layer from OpenGL/OpenCL/Vulkan/CUDA to DirectX. And don't forgot all the blotware and slowness that have Windows 10.
But then you lose all the benefits of an established desktop OS, like all hardware pretty much working, the OS pretty much working out of the box, plus software like Microsoft Office just runs without any additional effort, plus you get to profit from years or decades of muscle memory and OS-specific knowlegde, and maybe it's even about being allowed to connect your laptop to the company networks at all.
This is the first draft of the Microsoft Virtual GPU (vGPU) driver. The
driver exposes a paravirtualized GPU to user mode applications running
in a virtual machine on a Windows host. This enables hardware
acceleration in environment such as WSL (Windows Subsystem for Linux)
where the Linux virtual machine is able to share the GPU with the
Windows host.
So this isn't actual "DirectX on Linux", just a driver for a virtual GPU exposed to WSL-guests to enable guests to directly use DirectX, more or less.
This sounds like an attempt to win back ML segment from Linux to Windows. There are three letters back in my mind that sound like screaming, but too early to tell unfortunately.
It is an extension of the capability of WSL, giving you that sweet convenience of the DirectX API with your existing ML project. Of course, this extension makes your project incompatible with desktop Linux once adopted.
Not necessarily. It is not possible to access the GPU in WSL at all right now, so I need to dual boot which also means dealing with Linux desktop compatibility issues with my laptop.
As long as I can use the same ML framework (without DirectX API), then this poses no compatibility issues at all. It just means I can develop & run my ML code in WSL.
It's way more of a pain, due to a lot of legacy constraints in windows. It's easy to overflow pathnames and command lines that then get silently truncated. Doing ML dev work is for sure easier on a Linux env than a pure Windows env. it's not impossible on Windows, but def much nicer in Linux.
Except your ML project is accessing the DirectX API through another cross-platform API layer (CUDA). And the purpose of running WSL is that you ultimately hope to deploy to Linux servers (which Microsoft hopes will be on Azure).
The thing with WSL is that it doesn't win back the ML segment from Linux to Windows for production workloads. This is just a developer workstation friendly move that explicitly doesn't tie you to Windows itself.
It seems like Microsoft is continuing to see Linux as a production server target while positioning Windows to remain relevant as a workstation OS.
Ostensibly, this is a move to compete with Mac OS and not Linux.
I'm not too sure about that, what about the support for DX12 in WSL? Can't deploy that to production outside WSL right now as far as I'm aware, unless they add DX12 support to azure Linux vms...
Given that the main use case is stuff like running machine learning tools, the only people directly targeting DX12 on WSL will probably be framework/library developers, who might want to add DX12 as one of the graphics systems they support. If that's the case, the vast majority of developers won't notice any difference other than more software supporting graphics acceleration when run in WSL (or, another way of looking at it, when Linux is running on the WSL "hardware"/"platform").
Sure, nothing stops you from targeting DX12 directly in your application code, but why would you do that? At that point, you'd just target Windows since your users would have to be running it anyway.
> I'm not too sure about that, what about the support for DX12 in WSL?
It's an implementation detail -- they're exposing the Windows graphics driver to the Linux system with the most minimal amount of translation and overhead.
You could code directly to it in your Linux application code but it makes no sense to do that. You'd be literally writing a Linux application that can only run under Windows -- the smallest market ever proposed. Instead library/framework developers will add it as another target to improve performance in WSL for generic Linux applications.
No, because developers won't be coding to this DirectX module directly. They'll be using CUDA or OpenGL or using another library or framework for which this is just one of many backend implementations.
Unless you think developers would actually bother coding explicitly for the world smallest possible market (Linux inside of Windows).
Mentioned it in another comment, but I am not sure what stoped people from doing ML without WSL. I don't even know ML libraries, that don't work on Windows proper.
No, if you read the blog post on Microsoft's site it goes into more detail:
This is the real and full D3D12 API, no imitations, pretender or reimplementation here… this is the real deal. libd3d12.so is compiled from the same source code as d3d12.dll on Windows but for a Linux target. It offers the same level of functionality and performance (minus virtualization overhead).
Actually, the year of Linux on the desktop will never come also thanks to this. Developers will find more comfortable writing Linux software on Windows (some already do); desktop users will have both worlds at hand without being forced to dual boot; Linux gamers won't have any reasons to keep using native Linux. Etc.
I foresee in a not so distant future Microsoft integrating Windows UI elements and events hooks right into WSL, so that Linux developers too will be able to take advantage of the well standardized Windows UI; no more GTK or Qt frankenapps on Windows: just pure Linux code written on well known Windows RADs that access all windows internals while opening standard Windows dialogs and widgets. That would likely take away a huge part of Linux users.
I disagree, i do not think that anyone would want to start running windows just to run the same software they already run while losing a chunk of control and support. However for the real argument, ask yourself: am I switching to using windows because of this?
I think it's the other way round: Do I still need to dual-boot or switch to Linux if this works? Or can I stick with Windows and have the best of both worlds?
I do not agree with it being "The best of both worlds".
You do not control windows, it controls you. You must adapt to it being in your life and the choices made by its designers, this is both not secure and insanity for a software engineer to allow as an ongoing situation.
Even dual booting windows will mean its updater will overwrite the partitioning table of your drive and hide any other OS you have installed, and as I understand this people only do it out of need and not out of want.
I'm using Apple products for my computing needs, and I din't particularly like Windows, but Apple tends to be even more opinionated and restrictive – and I really like that a lot. It's a big feature of the ecosystem that I get a lot of pretty good defaults that work quite well, even if they aren't what I'd build from scratch if I had to. That way I don't have to expend energy crafting everything myself and can spend that energy on whatever it is I wat to do with my computer. I don't care about the exact type of steel used for my hammer, just give me a reasonably good one and let me build that thing.
In that way, MacOS currently is the best of both worlds for me – hassle-free if somewhat constrained, but I still have a UNIX underneath, iTerm, all sorts of utilities, and so on.
> the year of Linux on the desktop will never come also thanks to this.
I think you're giving MS too much credit here. Apple has one UI, and Windows has one UI (2 if you count windows in tablet mode). Linux has, what? 30?
Quoth the Torvalds:
"I still wish we were better at having a standardize desktop that goes across all the distributions… It’s not a kernel issue. It’s more of a personal annoyance how the fragmentation of the different vendors have, I think, held the desktop back a bit."
Only recently did Ubuntu stop trying to push Unity and accepted Gnome, thereby reducing fragmentation by 1.
I think that both Ubuntu and Red Hat family being based on Gnome is a good thing for Linux on the desktop, as there is a clear target for Linux GUI applications. I like the fact, that you still have the freedom to run any kind of desktop environment of Linux, but a stable default can only help Linux.
Looks like a very realistic scenario. From this prospective Linux is slated to become just another part of the MinGW world, thus having nothing to do with Linux as a potential end in itself. I would not be surprised if one day WSL is advertised as "the best environment for Windows programming."
This patch adds WDDM (Windows Display Driver Model) as a Linux kernel API. The implementation just forwards to the Windows kernel, but it seems to me like someone could implement it natively in Linux. Then Windows user mode graphics drivers would work natively in Linux without WSL, and DX12 too.
This is exactly what we don't want. There's already plenty of effort wasted on GBM vs. EGLStream so introducing a third API would just lower quality even more.
This is actually debatable. How many different GPUs are supported on Windows vs ob Linux? There are really only three vendors left on PC, while Linux supports these as well as a bunch of mobile GPUs...
This "third API" is already well supported by every relevant graphics vendor. Adopting it would actually reduce the number of APIs vendors need to deal with. And the quality of WDDM drivers has always been higher than anything Linux has.
Great, just what I want, vendors to be incentivized to force me to run closed binary blobs on my Linux box because they'll just repackage their Windows driver and still not release its source.
How? For example code that uses CUDA on Linux, calls the GPU CUDA driver, which passes it on to the Windows part of the interface. How does that require user code that doesn't run on normal Linux to take advantage of this?
If I understand it correctly, it just means that GPU-accelerated code that previously only ran on "proper" Linux now can run on WSL too, being powered by DirectX behind the scenes.
You don't understand it correctly.
Nobody is complaining or even talking about the gpu driver.
> DxCore & D3D12 on Linux
Projecting a WDDM compatible abstraction for the GPU inside of Linux allowed us to recompile and bring our premiere graphics API to Linux when running in WSL.
This is the real and full D3D12 API, no imitations, pretender or reimplementation here… this is the real deal. l
> libd3d12.so and libdxcore.so are closed source, pre-compiled user mode binaries that ship as part of Windows. These binaries are compatible with glibc based distros and are automatically mounted under /usr/lib/wsl/lib and made visible to the loader. In other words, these APIs work right out of the box without the need to install additional packages or tweak the distro’s configuration. Support is currently limited to glibc based distros such as Ubuntu, Debian, Fedora, Centos, SUSE, etc…
So you can use it with MS blessed distros and only when running under WSL2. You can't use it without WSL2.
> The plan is for Microsoft to provide shims to allow the existing Linux userspace interact with DX12; I'll explain below why we had to pipe DX12 all the way into the Linux guest, but this is not to introduce DX12 into the Linux world as competition. There is no intent for anyone in
the Linux world to start coding for the DX12 API.
Rather than "get people to write code that runs on WSL2 but not regular Linux," the goal is "get code that runs on regular Linux but not WSL to run on WSL2."
The developers' word is irrelevant. Microsoft is a business. Microsoft will pursue its long term business interests. Developers are hired to do what the business directs. So the question is really, "what would best serve Microsoft's business interests?" and not "what do the developers intend?" because in time only one question matters and unfortunately it's not the one with the best interests of non-windows-users in mind.
The developers' intent does align with Microsoft's current interests -- as you say, they are hired to do what the business directs.
Perhaps their interests will shift in the future, as they clearly have in the past. But the things they build now are not from the "extend" phase of an EEE arc. At worst they are in the "embrace" phase.
If MS make it as easy to run desktop apps on Windows as it is on Linux, then the question of why even run a dedicated Linux machine if you can do everything on Windows? becomes relevant.
I suppose clipboard integration already works? And drag n' drop?
Once that mindset is heavily entrenched (e.g. in Enterprises), then the extend phase can begin.
I don't think MS are as scared as they were previously when Linux started to dominate the server landscape, and swathes of new developers (especially web backend) moved to a java, ROR and Python (or LAMP), where MS were absolutely nowhere in the stack.
The initial implementation of WSL was not mandated by the business. it was created by the developers that were tasked with exploring how to run Android apps (on the ms phone) and they went bananas and invented and created WSL. First later the business side of Microsoft came in to play on wsl
It doesn't matter where it came from. WSL is owned by MS, and the most rational framework to predict how they will use it is that they will attempt to leverage it to support their business interests.
MS is not in the business of paying developers to give away nice things for free.
> Microsoft will pursue its long term business interests.
Just like all positive sum games, open source is in everyone's interests, including Microsoft's. Egoistic alturism: serving your own interests [through] serving everyone else's interests.
All the people here railing on how Microsoft is destroying open source clearly don't understand the very premise of the thing that they think they are defending.
This announcement is the evidence. They release some directx stubs that are completely useless outside of windows and of absolutely zero importance to anyone in the Linux Ecosystem and make a huge Microsoft hearts Linux push with it with confetti, unicorns and blushing dev-twitter influencers.
While the actual meat of directx will still be proprietary and windows-exclusive where they call the shots and make the money.
Releasing something exclusive is not evidence of EEE. Yes, it’s embracing, but that’s it.
How do you know they don’t plan to port DirectX to Linux? They’re currently attempting to upstream their changes, so why wouldn’t they port DX itself? They also don’t have to open source it (although I wish they would); they could just release a binary blob that you download and install.
Also, how do you expect Microsoft to kill Linux? To me, it seems that people who willingly choose to use Linux over Windows are generally the kinds of people who won’t be persuaded to move back. Microsoft can’t kill Linux.
Have you ever looked past your preconceived biases and considered that maybe Microsoft has actually had a change in culture over two decades? I don’t think so, because the people who scream “EEE” are the kinds of people who’ll never be swayed in their opinion.
> but it opens the door to code that runs on WSL2 that doesn't run on regular Linux.
Which is exactly what I was expecting since day one when WSL was announced. WSL is going to slowly kill Linux, or to be more accurate, it will kill any Linux (be it on servers or desktops) not Microsoft branded and distributed.
Forks on smaller embedded systems will resist for a while, until one day MS decides to port some killer technology de facto embracing them as well.
Yeah, but right now it solves the problem of not being able to run code on WSL that does run fine in Linux.
What really stops me from using Linux on the Desktop is compatibility issues with my laptop(s), reliable sleep/wake and rendering issues with High DPI display.
Microsoft released an MS Teams client for Linux which astonished me. So, I am giving them the benefit of the doubt for the time being.
Microsoft hasn't given up on Windows but having lost the mobile space I think they've decided to go back to their roots of developing software for many operating systems. It doesn't matter if you run Windows, Mac OS, Android, or iOS if they can sell you an Office 365 subscription.
The Windows protectionism that dominated the 2000's doesn't help Microsoft sell more product anymore.
It's not so much that Microsoft has changed but that the world has changed and Microsoft is changing with it.
The Teams client in Linux sucks, it is missing half the features and you have to pkill it after use, because simply closing it doesn't actually stop the program + it spins your fan like crazy doing God knows what in the background.
Agree. I am also looking for a laptop to mainly run Linux on. This quite widens the choices for me. Also, what keeps me and probably many others from running 100% Linux is the requirement of some commercial applications which are not available for Linux. So far using a Mac with a Linux VM was the best compromise for me, but now Windows becomes an appealing alternative.
I think Windows has been trying to kill linux for what, almost 30 years? Some of that conspiracy mentality may have been valid 15 years ago, but probably not anymore.
DirectX does run on regular Linux, courtesy of Wine/Proton. It's just a matter of reimplementing the userspace .so interface that MS will provide for access to this facility, so that it hooks into that support instead.
That's not the point. They are trying to not IBM themselves.
They and the clones won the war with IBM in the 80s and 90s because IBM was worried about things that didn't matter: they wasted time with things like an operating system CP/X86 that ran DOS and 3270 emulators to their mainframes on top of a GUI system called mermaid (3270 PC) while Microsoft was writing Windows 3. Oh yeah did I mention some configurations cost $20,000 each? some configurations cost $20,000 each.
Because IBM wanted to secure its AS/400 minicomputers, mainframe, and microcomputer line and make them interoperable, as if a PC user sitting in their den would be connecting with $500,000 or so of other IBM machines, everyone else ran circles around IBM. They kept up the "whole kitchen sink" system way way past its due date.
This also happened to DEC, which responded way too late in the game to be relevant, SUN, SGI, and Wang. And it may eventually take down Oracle.
So they're intentionally taking an uncommitted, decoupled approach. I'm paying Microsoft as a result. I pay GitHub and Azure bills every month and run exclusively Linux.
They can't sway around as a monopoly like they could in 2000, once Ballmer left things changed rapidly. It's not going back. They have effectively zero cloud software (database/operating system etc), effectively zero mobile presence, and most people don't really like Windows.
In a tightly coupled stack you're only as strong as your weakest link, and MS has a bunch. Their biggest risk now is to HP or GM themselves; basically gobble up a bunch of things, blend it into indifferentiable blandness and collapse while waving a giant sceptre labelled "greatest company of 30 years ago" (HP bought DEC, Cray, Tandem, Apollo, Convex, 3Com, Phoenix, Palm and SGI and did effectively nothing with them beyond slapping an HP logo on their final pre acquisition product line and then just rode it out without any followup. HP knowing only how to fumble the ball every time for 20 years is why big business switched to Linux. HP took all the alternatives behind the barn one by one, cut them a fat check and then shot them. Simply crazy)
Hopefully, this will enable Windows users will be able to run docker containers that use GPUs the way Linux users have been able to for years now with nvidia’s container toolkit[1].
From the linked blog post [1], it sounds like it: "In addition to CUDA support, we are also bringing support for NVIDIA-docker tools within WSL. The same containerized GPU workload that executes in the cloud can run as-is inside of WSL."
This is the kind of driver that you get when you ignore all existing Linux code, then try to throw 15k LOC over the wall when its all done. Microsoft should really know better: this is not how you get your code upstreamed.
I doubt Microsoft really cares. Microsoft will ship it in the linux kernel in WSL2, and people will use it, and if the upstream community doesn't want it I don't see why Microsoft would have a problem with that.
They'd certainly prefer that upstream Linux maintains it for them. The more code they have to maintain out of tree, the harder it is for them to upgrade their WSL kernel.
You're not seeing the big picture. If Windows can do Windows apps and also 90% of what Linux developers use Linux for, what need is there ever for Linux on the desktop? MS marketing may be trumpeting their love for open source but WSL2 is a actually a brilliant bit of jujutsu[0] on Microsoft's part.
[0] I use this word specifically because it's a martial art about using an opponent's strength against themselves. MS's use of Chrome as the basis for their new browser is another example of this.
Microsoft saw developers flock to Mac OS/X where they had a true terminal and could also run line of business apps like Office. I was at several Microsoft developer conferences where 90% of the machines were MacBook Pros and I know that didn't make Microsoft leadership happy. I believe the Surface line was a direct response to that along with WSL. Someone in Redmond realized they couldn't beat Linux so now comes the embrace.
Someone please correct me if I'm wrong but I interpret this as being only a bridge to DirectX from the Linux kernel. Not a DX implementation on Linux.
You won't be able to take this dxgkrnl driver and load it into a Linux workstation and get DirectX.
And also, someone else responded to you what's the point of running Linux if Windows can do everything Linux can.
I would assume that most full time Linux users are using it out of at least a tiny bit of ideological motivation. And if not ideological then habitual, having had the power and granular transparency of Linux for so many years that Windows would never be an alternative.
> This means that code written for this will NOT function on desktop Linux.
Like CUDA, I assume that most applications would not be coded to this API directly. Microsoft already mentioned OpenCL and OpenGL.
This is to hardware accelerate Linux applications and not to create WSL-specific Linux apps. I can't imagine there's a big market for Linux GUI apps that would only run on Windows.
Maybe I'm missing something but how is this an improvement over virgl? Something that's platform agnostic and works with applications right now being replaced with something that's very tied to windows.
Thank you. In the meantime I also noticed the following comment from Dave Airlie on LKML [1]:
> This is a driver that connects a binary blob interface in the Windows kernel drivers to a binary blob that you run inside a Linux guest. It's a binary transport between two binary pieces. [...] I can see why it might be nice to have this upstream, but I don't forsee any other Linux distributor ever enabling it or having to ship it, it's purely a WSL2 pipe.
That being said I don't see why distros like Ubuntu that allow non free code wouldn't ship with it out of the box. They already support Nvidia binary drivers anyway, so why wouldn't they support a transport bridge that'd let people use them on WSL too?
Also, he notices that it's intended to be more than a bridge in the future, as they're already working on integrating it with the presentation layer on Linux.
Why would they? In its current state, this driver is only useful when Linux is running on Windows. A regular Linux desktop/server user would not find this useful.
Azure is more than 50% Linux. There's an incredible amount of "regular" Linux instances running on top of hyper-v. This lets them use a virtualized GPU in that env.
On top of that, why would canonical care what hypervisor you're running? There's so many other cases where they haven't gone "eww.. proprietary", why would they start now.
I don’t think azure is relevant here. Standardizing SR-IOV would be much more helpful to everybody (but much more important to Azure/cloud platforms). Kvm setups can already pass through gpus just fine.
This is a way to lock developers who want to run Linux code for AI/ML onto their windows machines and to keep them from switching their desktop OS to Linux.
Edit: from the kernel mailing list
> There is a single usecase for this: WSL2 developer who wants to run
machine learning on his GPU. The developer is working on his laptop,
which is running Windows and that laptop has a single GPU that Windows
is using
> I don’t think azure is relevant here. Standardizing SR-IOV would be much more helpful to everybody (but much more important to Azure/cloud platforms).
Meanwhile today, Nvidia and AMD GPUs don't support SR-IOV. So that's a non-starter. It's also not clear that it the right model for GPUs either, as they have their own MMUs already.
> Kvm setups can already pass through gpus just fine.
But you need to dedicate that GPU to the guest and the host can't use it anymore.
>> > There is a single usecase for this: WSL2 developer who wants to run machine learning on his GPU. The developer is working on his laptop, which is running Windows and that laptop has a single GPU that Windows is using
From the current patch set, because they haven't hooked up the swap chains yet (as they've said that they're doing) which would allow full graphics support in the guest.
Because this benefits only WSL. I don’t think that the Linux maintainers should spend time and effort maintaining a shim that only benefits users on another operating system. It does not even lead to cross platform code if a dev uses this on WSL.
Microsoft can ship it however they please though within bounds of licenses.
Probably because its a shim to use a proprietary (only distributed by MS) binary blob version of DirectX compiled for Linux. Outside of ML utilizing DirectX in WSL on Windows, the only group that this helps is MS.
FTR, I'm not arguing whether or not this should be upstreamed. I just see where the other poster could be coming from. I could be wrong on my take and if so, someone please correct me.
VMWare is probably the company with the worse reputation than Microsoft since they spent more than a decade shipping large part of Linux linked against their proprietary hypervisor in a blatant GPL violation.
Microsoft has a history of anti-competitive practices that crippled the industry. In either case, serious objections are justified and severe handling of one case doesn't invalidate the handling of the other.
There's been Microsoft levels of anticompetitive practices from every proprietary hypervisor vendor. The others are seen as "oh, look! They're working with Linux! yay!" for a code dump across the wall (like vmware, and arguably z/VM), whereas here Microsoft appears to be doing everything right and working with upstream to modify a minimal MVP and are being lambasted for it.
It feels like they're doing everything right and everything we've asked for, for decades, and being shunned for it. What's the mechanism here for Linux winning and Microsoft being allowed into the community and what should they be doing different?
They're not doing everything right. They're doing something objectionable, like other people are also doing. Except they're being called out for it because of their reputation of having one of the most hostile anti-competitive practices the industry has ever seen. It's the price you pay for having such bad credit, even if your intentions initially seem benevolent.
> They're not doing everything right. They're doing something objectionable
What exactly? The only performant way to expose GPUs to guests while still allowing them to be used by the host (or multiple guests) for GPUs that don't have hardware support for partitioning themselves (ie. pretty much all discrete GPUs) is to replicate the ioctl layer from the host up into the guest. Any other solution is a non starter if you care about getting the perf you'd expect from the paravirtualized GPU.
We've changed from https://lkml.org/lkml/2020/5/19/742 (and that title) to the blog post that it points to (and its title), which presumably makes things clearer.
OpenCL, DirectX and CUDA all seem to be available for GPU acceleration on WSL.
One other exciting thing is GPU accelerated GUI Linux apps on Windows. You may not need an X server anymore. I'm looking forward to using i3 on Windows.
This isn't DirectX on Linux. This is DirectX API access for WSL exclusively. It will lock ML projects into Windows/WSL for the price of access to the DirectX API and at the expense of development of other projects like Vulkan for native Linux.
I don't see how being "locked into CUDA" should make one less skeptical of a change that will functionally lock projects into Windows. I can't help but think this is going to siphon resources from Linux native graphics development.
> The plan is for Microsoft to provide shims to allow the existing Linux userspace interact with DX12; I'll explain below why we had to pipe DX12 all the way into the Linux guest, but this is not to introduce DX12 into the Linux world as competition. There is no intent for anyone in the Linux world to start coding for the DX12 API.
* They create a new kernel file descriptor: /dev/dxg
* With this, on the Linux of WSL, it is like a direct "pipe" to a Windows host graphical stack.
* So it means that they can put code in Linux application code that will use Windows proprietary graphical stacks, like DirectX through /dev/dxg.
* And so, clearly, this Linux app will not work inside a normal Linux computer that is not a "guest" of Windows.
Now, you can see the "Embrace" and the "Extend"?
And to well understand the article, the following things have to be stressed:
This is not just something to render a Window or something like that, it is a special "passthrough" api to all the things that are provided by Windows GPU stack/drivers.
For example, they give the example of Cuda compute API, but also, they kind of "built" DirectX sdk itself for Linux (debian, ubuntu, ...), but still the proprietary closed blob that "apps" are expected to use, but that will rely on /dev/dxg.
Also, if you want to use anything OpenGL on the Linux WSL, they will ensure that it is translated to DX on Linux side, before going through the same special DirectX api.
The good thing would probably have been to do the opposite side, ensure that Windows has the proper OpenGL support to be able to pass gpu acceleration to the Windows host.
If this works, Microsoft will overnight become viable for scientific dev work. Kudos to MS for taking on what is perceived by many as a "fringe" feature, but would allow me to use Windows on a laptop, while having a sane (and compute-rich) Linux dev environment. In all these years, NVIDIA still didn't make it easy to use laptop GPUs from Linux directly. At least not if you want good battery life.
There is a new, closed-source Microsoft DirectX 12 library for Linux apps that speaks WDDM to /dev/dxg.
There is a pre-existing closed-source NVIDIA Vulkan+OpenGL+CUDA library for Linux apps that speaks EGLStreams to /dev/nvidia0.
There is an announcement that the closed-source NVIDIA library might start speaking WDDM to /dev/dxg.
There is a pre-existing open-source Mesa DirectX 12 (+Vulkan +OpenGL + ...) library for Linux apps that speak GBM to /dev/dri (and some support for speaking EGLStreams to /dev/nvidia0 too).
There is a pull request to implement /dev/dxg in the kernel as a Hyper-V pipe for WSL2's use.
There are lots of interaction points for alternate implementations of various pieces of the stack. For example, in the future it might ultimately be possible for an alternate /dev/dxg implementation to wrap the normal DRM API, or for NVIDIA's binary driver to reimplement /dev/dxg on real hardware.
I really wish Microsoft would just tell NVidia (and AMD) to support SR-IOV on all their GPUs. Then we wouldn't need any major software changes to enable CUDA acceleration within VMs, just a configuration change to pass through a virtual function of the GPU to make it available to existing drivers.
AFAIK that's up to motherboard manufacturers. SR-IOV often doesn't work on desktops, I doubt there mass produced notbooks support it at all. MS can work with manufacturers, but they have their own incentives and it is unlikely to expect that all of them suddenly start doing a good job on this front.
Meanwhile MS wants to cast widest possible net, so it needs to enable more and more workflows on WSL2 today, not tomorrow.
Other than enabling the IOMMU (which any system with Thunderbolt should have on by default), what does the motherboard firmware actually need to do for SR-IOV? I can't see any reason why it would need to be concerned with enumerating Virtual Functions or anything like that, but I don't actually have the SR-IOV spec on hand to dig through.
EDIT: Looking through an old Intel whitepaper, it looks like the system firmware at most has to reserve some extra config space for SR-IOV devices when enumerating the PFs, so that the OS can enumerate VFs after creating them. But Linux includes an option to re-allocate this stuff if the BIOS doesn't reserve space, so this apparently isn't a hard requirement.
SR-IOV doesn't make sense on most GPUs, because they have their own MMUs already with different semantics. SR-IOV is terrible for dealing with a large bank of RAM on the target device, because the MMU is in the wrong part of the architecture. It's all the way on the root complex, so all GPU VRAM reads and writes would need to take the slow path through PCI-E and back. This is basically a nonstarter for GPUs that have their own VRAM. They also don't cover the GPU specific caching information that's normally stored in the GPU page tables either.
Intel is able to get away with it for integrated GPUs because they can codesign their system level IO-MMU with the GPU MMU and it doesn't have it's own VRAM bank. I'd bet dollars to donuts that Intel's new Xe discrete GPU doesn't support SR-IOV even though the integrated versions of the same core will.
It's not really SR-IOV in the way you'd generally consider it as it only virtualizes the GPU's access to main memory not VRAM. That still requires that you setup GPU page tables and manage them on the host side with a bridge driver in the guest like Microsoft's work here is doing.
547 comments
[ 3.2 ms ] story [ 102 ms ] threadDollars to donuts this is why Microsoft is implementing this. GPU acceleration is becoming a critical feature for many users (but especially developers) and this will continue. If WSL is to be a serious competitor, this is necessary and I'm glad to see it showing up. This is true of cloud compute, too, and Microsoft is betting big on cloud as its future growth area.
> Only the rendering/compute aspect of the GPU are projected to the virtual machine, no display functionality is exposed.
The Linux gaming folks will be pretty sad about this one. Anyway, this isn't really a Linux port of DirectX. This is GPU compute via DirectX APIs.
So now, I'm just waiting on monitor mode/AF_PACKET for WSL...
And yet another piece is a layer to get OpenGL and OpenCL workloads running on DX12 as well, rather similar in scope to how MoltenVK and the gfx-hal Vulkan Portability work are a layer to get Vulkan workloads running on Metal. This is a big effort, and it seems to me their goal is to get things to the point where stuff Just Works and you don't have to think too hard about the various bits of (technically difficult!) infrastructure to get you there.
Huh? Are you sure about that? Regular TensorFlow on Windows uses CUDA, not DirectX-flavored compute.
> Anyway, this isn't really a Linux port of DirectX
The entire user mode side of Direct3D is ported, in addition to the user mode parts of the Nvidia, AMD, and Intel graphics drivers.
Are we seeing the start of the migration of Windows to linux?
Please no. Please keep your peanut butter out of my chocolate. Call me a purist, but linux should take nothing from windows, give no ground, make no compromise. One must die for the other to live.
https://bugs.launchpad.net/ubuntu/+bug/1
Edit: yep, an online search seems to say that's an actual thing. I guess I'm part of the ten thousand today https://xkcd.com/1053/. I will never understand the US fascination for peanut butter.
At one point in history, US farmers were encouraged to grow peanuts as a rotation crop to improve soil quality. That led to a glut of peanuts in the market, so people tried to find uses for them. Peanut butter was invented+ as one of these uses, and has been a staple of American diets ever since.
+Or promoted, I can’t remember
https://aces.nmsu.edu/pubs/_a/A129/
[0]: https://commons.wikimedia.org/wiki/File:Reeses-PB-Cups.jpg
And yes, as a sibling notes, Reese's peanut butter cups are actually alarmingly tasty, but.... as with any $1 chocolate bar, that's shitty HFCS-saturated chocolate and shitty palm-oil-laced peanut butter, with way too much sugar in it, so if you're too good for that, well, that's a credit to your tastebuds, good on ya.
So eat real chocolate with real peanut butter. Real peanut butter is nothing but peanuts and salt (it keeps well, but fresh-ground is better). Real chocolate, I trust you can figure out. Milk and dark are both good in this application.
Although, of course, peanuts are not true nuts (no more than macadamia or almond or walnut), they're nonetheless very nutty, and the effect is pretty similar to "almond bark", or hazelnuts with chocolate, or pecans and chocolate. And of course you can just eat peanuts with chocolate, an okay combination. But there's something weirdly perfect about peanut butter with chocolate, better than peanuts with chocolate.
But hey, although I'm not American, I am from America's hat, and I do like peanut butter in a few other formats too.
By making the virtualized hardware the "glue", they can avoid the GPL/copyleft infection of their commercial OS, while supporting different kinds of developer experiences.
Though in principle, there's nothing preventing them from using GBM instead of EGLStreams, and there are some good practical reasons such as having compatibility with the broad base of existing accelerated Wayland windowing libraries and applications.
The people working on the NVIDIA open source driver have no official support and were fighting with signed firmware blobs last I heard. I wish them luck, but even on older cards it is more likely to crash your system than render anything.
My understanding is that even 2015 is too new for nouveau to run with high performance due to something called reclocking, where the card starts up at a minimal clock rate and then it's up to the drivers to reconfigure it for running at the advertised clock.
As if describing things in terms of northern hemisphere temperate seasons wasn’t bad enough (and still worse commonly showing how little you care about any place other than the USA and maybe Canada by using the name “fall”), now we have this: “holiday 2020”. I don’t know when this is talking about. I’d have guessed northern hemisphere summer school break first, but I guess that’s just about finished now, so it can’t be that. Christmas time would have been my next guess, but surely you’d describe that as “by the end of 2020”? And then other possibilities occurred to me—Halloween? Thanksgiving? I have no idea at all what Americans would call “holiday” as a time of year.
They're referring to https://en.m.wikipedia.org/wiki/Christmas_and_holiday_season
> I’d have guessed northern hemisphere summer school break first, but I guess that’s just about finished now
Summer break more or less just started for most students here. The fall term will start around August.
> There is a single usecase for this: WSL2 developer who wants to run machine learning on his GPU. The developer is working on his laptop, which is running Windows and that laptop has a single GPU that Windows is using.
Can't say I can get behind MS trying to shift maintenance for a Windows only "feature" onto the Linux devs here.
1. https://lkml.org/lkml/2020/5/19/1139
And if it is rejected, Microsoft can still ship it in the kernels for the distros they offer on WSL.
Frankly this does just seem like MS wanting to reduce their maintenance burden on what they expect will be a very important part of their WSL offering on Windows. There's nothing inherently wrong with that desire, but the people on the other side need to weigh their maintenance burden[0] with what benefit this will have to the Linux community as a whole, which at first blush seems minimal. Especially considering that the userland pieces that talk to this driver aren't open-source.
There's also the question of whether or not you believe WSL as a whole is good or bad for Linux. If there are people who would run a Linux desktop for development who then decide not to because WSL exists, perhaps that's a bad outcome. If you have people writing more DirectX GPGPU code who would otherwise write to a standard interface like OpenCL, perhaps that's a bad outcome (to be fair, there's also a lot of CUDA out there, which is similarly problematic). Is this the start of MS going back to their "Embrace, Extend, Extinguish" playbook, or is that just a paranoid fear? They've definitely been embracing Linux, and enabling people to write DX12 GPGPU code that targets a Linux environment but will only run under WSL on a Windows install does feel like "extend".
I'm not sure where I personally stand on this issue as I haven't done my research, but I think they're interesting questions to ask.
If this gets rejected, of course it doesn't stop MS from doing any of these things, but it does make it harder for them to maintain their extensions to Linux.
[0] Airlie is even concerned that just by looking at the code, he or other DRI developers could run into future IP derived-works trouble when designing future Linux graphics interfaces.
I think if that really matters you should be against anything in the kernel to make it work well as a VM under Windows or MacOS or BSD, and in VMware or VirtualBox too.
From that point of view, Linux in a VM on another host is taking away from Linux running as a desktop on that host. Linux running as a VM on a VMware cluster is taking away from Linux running on those bare metal servers instead of VMware ESXi.
I think the more sane way to look at it is that Linux is an application (and its subcategory is that it is an OS) which is meant to run on various hardware and software platforms, the more the better. This strategy has worked very well over that last couple decades.
Does allowing WSL mean that some people that would install Linux on their hardware just run Windows instead and use Linux on top? Probably. Does it mean that people that already ran Windows and have never used Linux get exposed to it for the first time and get familiar with it through a few click on their existing Windows computer? That's also probable. Does it really matter in the end? Probably not.
Why? I would guess that a very large share of linux kernels run under a hypervisor in some data center, in a public cloud or some OpenStack cluster. Won't those mostly be the same features?
>>> I think if that really matters you should be against
Should be interpreted as (and I meant to write as)
>>> I think if that really matters to you you should be against
The rest of the comment should have made that obvious though, especially the last two paragraphs.
Happy to know that we also agree then.
Given that the WSL2 rewrite is essentially this, without even the niceties of a VirtualBox GUI wrapper to control the settings, I keep wondering what all the fuss is about.
IMO it's a good thing. Given that windows accounts for 90%+ of the desktop OS share, Windows might very well become the world's most used Linux distro.
I can't even say I wouldn't use it - it might be nice! But I will not use any WSL-only capability, that's for sure.
I'm still piecing it all together, and I definitely feel that "Extend" feeling, but I'm not sure that's what's happening here. Looks more like a few devs at MS are trying to solve the GPU Accel use case for WSL...
Those tools and libraries will then not work on native Linux.
WSL2 literally runs user-mode distros (and their binaries) in containers atop a shared Linux kernel image (https://github.com/microsoft/WSL2-Linux-Kernel) inside a lightweight VM that can boot an image from cold in < 2s and which aggressively releases resources back to the host when freed.
So when you run a binary/distro on WSL2, you are LITERALLY running on Linux in a VM alongside all your favorite Windows apps and tools.
If some of the tools you run within WSL can take advantage of the machine's available GPUs etc. and integrate well with the Windows desktop & tools, then you benefit. As do the many Windows users who want/need to run Linux apps & tools but cannot dual-boot and/or who can't switch to Linux full-time.
This will (and already has) resulted in MANY Windows users getting access to Linux for the first time, or first time in a while, and are now enjoying the best of both worlds.
So people who use it are married to Windows.
I think folks would be absolutely excited if this was an initiative to allow writing DirectX applications on Linux, and available for Linux on bare metal. But as people realize this marries them to Windows, they go meh.
If DirectX on Linux could also work on bare metal, the conversation here would likely be different.
And that is, IMHO, a very real concern and it really should not be merged.
I doubt the linux devs ever see WLS as a target they have to maintain themselves.
Using that with wine would mean adding two emulation layers before reaching the actual driver. I fail to see any use case for that.
So yeah, vulkan is neat and opens up a whole lot for the linux world. In the future you'll probably see userspace implementations of opengl on top of vulkan, maybe even CUDA implemented on AMD gpus, although I'm not sure how practical that is. Also a whole lot of exciting GPU sharing tech, accessing GPUs inside of VMs for example.
DirectX will come to linux, but it won't be thanks to microsoft. You can thank valve hedging their bets on the microsoft store for that.
The only time I faced the need for Linux box is trying a demo project from OpenAI, which did not use the features it required Linux for on a single machine anyway.
Doing something custom on top of the GPU is also not much different on Windows, than Linux. CUDA is basically the same. OpenCL and Vulkan are available too.
I'd like to hear perspective of a person, who actually does ML specifically on Linux for some reason.
Also windows not having a build in C compiler makes you dependent on the horribly convoluted Visual Basic stack that seems to have a lot of dependencies for some python ML libraries. Docker makes it a lot better to run and I almost always deploy in a docker container because the ML modules I deliver are often interacted with as a black box with a REST API on top.
You might be right about C compiler. But something itched when you mentioned Docker. Could getting Windows SDK installed be harder, than installing Docker?
That being said I had to avoid installing VS 2019 for quite awhile because Node.js native module build chain couldn't work with it. There are complexities
Even popular libraries like zeromq don't support namedpipes on windows because of how complicated they are and how different to everywhere else.
Just determining what visual studio version is installed seems to trip up projects all the time.
My main reason: It is the most convenient way to have Unix tools (grep/sort/cut/sed/less/...) and bash available. Cygwin always was a pain, MinGW / GitBash felt much better, but ultimately WSL just feels best.
These tools are incredibly valuable to my workflow. Sure, stuff like pandas can be nice for small datasets, and some data sits in some DB/Kafka/distributed system. But there have been countless cases where unix tools allowed me to take xxGB zpfiles of text and do basic examination or even build baseline models within a few hours.
Sure, there always are alternatives to use these tools and there are many equivalents. But I would always prefer WSL + conda for Linux to a typical "Windows Conda" installation with that weird GUI and the need to install so many different applications to even just look into the first or last few lines of a huge textfile.
EDIT: That said, of course I can/could always just run a juypter notebook under windows using windows cuda + GPU and share files with a WSL bash where I do my modifications. But again, everything within the same systems just feels better (ipython shell magic, no worries about if paths to the same file are really identical, etc) and while this is by no means a game-changer, it is just nicer that way.
https://github.com/pytorch/pytorch/issues/37790
weird
https://github.com/pytorch/pytorch/issues/32575
bug
https://github.com/pytorch/pytorch/issues/25301
I don't have to spend time explaining or justifying or isolating my setup.
Conversely though, my work is itself off the beaten path enough that I'm likely to run into weird bugs. If I was pushing images through a CNN, that'd be well-trodden enough on every platform that I'd be a lot less fussed about which particular platform I use.
I do not think it's purely windows to blame here though. It's only quite recently that NVIDIA started fixing their documentation and instructions on getting all the right CUDA CuDNN stuff running properly on a system.
Imagine if you could run AI/ML apps and tools that are coded to take advantage of DirectML on Windows and/or atop DirectML via WSL.
Now you can run the tools you want and need in whichever environment you like ... on any (capable) GPU you like: You don't have to buy a particular vendor's GPU to run your code.
If you're old like me and remember the dark ol' days when games shipped with specific drivers for (early) GPU cards/chips, but failed to run at all if you didn't have one of the supported cards, you'll understand why this is a big deal.
Maybe I'm not that old, but I'm old enough to remember the days when microsoft was intentionally degrading opengl performance on windows ;).
Those days sucked. Bigtime. If we can avoid doing the same mistakes for machine learning then we should.
Which is still nonsense, since this only affected the OGL driver shipped by Microsoft. In contrast to truly bad actors like Apple, OEM were free to ship their own OGL drivers from day 1.
So sorry mate, but I have to call BS on that one.
Don't you think the effort to achieve this would be absolutely massive? I don't know what kind of resources are thrown on this project, but I'd estimate minimum to be 3 dev teams for 2 years just to get a few variations of ResNet to work "as is". And that's just for regular models, that don't require quantization or (auto-)mixed precision for training.
Isn't the linked post saying you have to be running on Windows though? It seems like it would make way more sense to either port directX to Linux, or ditch directX and put those resources into supporting Vulcan.
o-O nvidia-docker does not even support Windows.
I think the only thing you need to know is which CUDA version your cuDNN requires, and it was quite clearly stated on the download page. Also the same on Linux. For nvidia-docker you used to need a specific driver version.
https://www.protondb.com/
Not sure why, but I suspect that Valve has a lot to do with it.
Conversely, if you run Windows, it's rare that you need to work hard to run a game.
But there is another way there - running Windows in a VM with GPU passthrough - works beautifully in my case.
Like the previous response what does this buy me as compared to developing and running natively in windows - as there are native compliers that support cuda etal on windows.
Check out https://old.reddit.com/r/VFIO/
I guess this will be the standard until we can have nicer graphics drivers for Linux.
That's not the usual situation for people that are trying to use this scheme.
If you have a spare GPU, a VM with PCI passthrough does an even better job, except for some anti-cheat software that artificially discriminates against this setup.
In theory it ought to be possible to switch a single GPU to/from a VM without a reboot. In practice I have no idea how huge a refactoring to the Linux graphics stack that'd require.
This has been true for 16 bit games since long before Windows 10. Ages ago one of my favourite games stopped working on Windows, but Wine had no problems with it. So my impression has always been that Wine is excellent for really old games, but slightly more recent games, it could already be very hit and miss.
> "If you have a spare GPU, a VM with PCI passthrough does an even better job, except for some anti-cheat software that artificially discriminates against this setup."
Doesn't every CPU these days have onboard graphics? My Thinkpad X1E should support hybrid graphics, so it'd be nice if I could give the GPU to a VM and have the desktop use the CPU graphics.
But if a Windows VM does a better job, that means Wine doesn't yet do as good a job as Windows. Though it's certainly true that Steam support for Linux is growing. But I don't think every Steam game already works on Linux.
On laptops, pretty much. On Intel desktops, yes, aside from Xeons. On AMD desktops, only some lower end Ryzens have "G" models.
Official Proton "support" is limited, because it requires certification by Valve and/or the game developers that the game runs well (the equivalent of a "native" rating on winedb/protondb), but if you're willing to go down to "gold" levels of support it still runs 70-80% of all Steam windows games.
See https://www.protondb.com/
Don't bother attempting GPU passthrough on any laptop with an AMD CPU (eg Ryzen 2700U) and Radeon GPU (eg RX 560X).
It turns out the GPU passthrough needs a dump of the Radeon BIOS provided as a file, but no-one can dump the BIOS of discrete AMD laptop GPUs. :( :( :(
Note the complete lack of RX 560X BIOS's here:
https://www.techpowerup.com/vgabios/?architecture=AMD
I've got this working today. I do it through swapping the nvidia driver for the vfio-pci driver (and back again if required). The slight annoyance is that you may need to restart X11 (for me this is not an issue).
I wrote about this some years ago: https://me.m01.eu/blog/2016/05/pci-passthrough-vm-monitor-se...
(My gaming desktop, of course, runs Windows.)
On laptops, however, I continue to find myself jumping through idiotic hoops to use Linux. The driver support is always just-barely-good-enough. Maybe it's audio, maybe it's graphics, maybe it's power management. Maybe it's something involving networking-after-power-management or some crap involving "don't unplug your headphones while the lid is down". On my current work laptop, a beautiful Thinkpad Carbon X1, I can't get the power management to work properly, so I just have to accept that there's no hibernate. I'm constantly forgetting to shut down, put it in my backpack, and then pull it out drained. What a pain in the ass. Could someone fix this problem, probably. Can I? Not in the dozens of hours I've put into it. I hate doing IT, I hate it I hate it I hate it.
However much Macs make me want to vomit in my mouth, I can see the appeal. The drivers work at least 90% as well as Windows drivers, and the UX is at least half as good as a lightly-tuned Linux machine. "Jack of all trades, master of none, is oftentimes better than master of one"
Anyway, before this lockdown ends, I'm upgrading my laptop distro to this new distro I've heard of out of Redmond, I think it's called "Windows".
Not to mention that there are quite a few development environments for more obscure platforms that still only exist for Windows.
Overall, since most development time is spent in an IDE, the OS is really of little relevance to software development. Sure, some people insist in using command line tools, and that is unlikely to be pleasant on Windows, but a lot of other developers don't, and we couldn't care whether we're running our Emacs on Linux or Windows or Genera or whatever.
However, if I'm using IntelliJ or Emacs and Firefox, I don't really need to care what OS is running underneath too much.
Edit: of course, Linux and Mac are available for devs that prefer them. It's still much easier for IT to manage 7000 Windows desktops and a couple hundred Linux ones than it would be to manage 7000 Linux desktops.
- Could you push a patch to Linux systems and have it install at the user's convenience (with some end date)?
- Can you do that in waves without manually configuring things?
- Can you remotely wipe a system if required?
- Is there any popular anti-virus software for Linux, to protect company files in home folders from user mistakes?
- Can you help users install some software without giving them full access, but also without requiring IT intervention for every installation?
Why? I have never seen Windows being managed entirely hands-off whereas Linux just works.
Where does the complexity on Linux come from that makes managing them more difficult?
Then there's the question of pushing an update to all managed computers. Maybe it's not a package update, but you want to change some SELinux policy for all users, or update some DNS server or the default search domain and so on.
Never mind the question of how you can instruct one of those Linux computers to delete all data it holds whenever it next connects to the internet (to handle the case of a stolen company laptop).
There are so many things that you need in an enterprise setting that have common (though probably quite expensive) tools available for Windows. Maybe some of these exist for Linux as well (I would expect RedHat to have some), but I'm not sure. Linux admin is usually reserved for servers much more than desktop computers.
Interestingly, apples have to be compared to oranges. On Linux, it is easy to identify the programs that are using a library. Thus it is easy to restart just the services that are patched. In general, things can be scripted so there are no tools available. But this requires somebody who understands the system. From a business perspective, this might be more expensive, or not, if the tools are expensive.
I use a Windows laptop at the company I currently work for, because everything is locked down and I wouldn't be able to get my own laptop connected to the network. (Or so I thought; I co-worker managed to use the Windows laptop as a bridge to his own Macbook.)
Now you're right that as long as I stay in the IDE, it's not so bad. But every once in a while I need to do something outside the IDE, and I immediately get slapped in the face by how stupid some things are. And because it's an enterprise environment, some things are even worse than usual; opening a folder, or saving something, can be unreasonably slow because either it's a network drive or it needs to be checked for viruses and malware while I'm trying to use it. Or for some other reason. I don' t know, I just experience the extreme slowness.
Also, on top of the old terrible DOS shell, there's now also a Power Shell that's supposed to be better. It apparently has some powerful features I don't really grasp, but it's still not remotely as good as bash. And sometimes the command line really is unavoidable.
But the real pain is at home. When I activate Windows 10 on a new machine, I need to create a Microsoft account. I don't want one, but it takes serious determination to avoid it, because behind every message is another trick to sucker you into an MS account. When you finally do manage to create a local account, you're immediately expected to compromise your security with 3 insecurity questions, and no way to avoid it as far as I can tell. Previous versions of Windows did not have this stupidity.
Also, somehow Windows keeps losing my mic, speakers or camera. Once I've found the right troubleshooter, it immediately figures out how to fix it, which is great, but it also keeps losing them again. And finding the right troubleshooter takes a couple of steps and a bit of searching. I feel like I need to pin several relevant troubleshooters to the taskbar.
And then there's the total lack of access control. To install anything, you need to be admin. I gave my son a restricted account, but he can't do anything with it. I'd like to be able to create an account that can instal games, but can't compromise the system. No such option in Windows. If you can do anything, you can do everything. Unless you're in an enterprise environment, in which case you often still can't do anything. So I guess more detailed access control does exist, but only for enterprise users or something.
> And then there's the total lack of access control. To install anything, you need to be admin. I gave my son a restricted account, but he can't do anything with it. I'd like to be able to create an account that can instal games, but can't compromise the system. No such option in Windows. If you can do anything, you can do everything. Unless you're in an enterprise environment, in which case you often still can't do anything. So I guess more detailed access control does exist, but only for enterprise users or something.
Here I never understand this point. You can't do anything on a Linux system if you don't have sudo access - it's not like apt or yum have any special magic to allow non-admin users to install stuff. And if you can install software on a system, you can already do anything else. Especially Games, which install drivers to perform DRM and anti-cheat bull.
Now, if you want to look into it and waste quite a bit of time, Windows does allow you to configure access control at a very fine-grained level for access to non-system folders. But as long as the installers want to install things in system folders, there really isn't any solution.
aka wine / proton
How do you configure compiler to use libraries downloaded from vcpkg? CMake? Something else?
Interesting take on the situation. This is effectively a driver they need to get into the kernel (just one that targets a paravirtualization host and not “real” hardware), and Linus has been adamant that the correct way to write a driver in Linux is to upstream it into the kernel.
The perspective that upstreaming a driver into the Linux kernel is a burden for Linux kernel developers is one I haven’t heard before, and seems to clash with Linus’s typical stance. Is this something that has some prior examples? Genuinely curious.
Just run a Linux hyper-v vm. That's what WSL2 is doing under the hood anyway. I run it this way and it's great. I have windows terminal auto ssh into it. Performance is great. And using the X server x410 on the windows side gui performance is fantastic (though no hardware acceleration) because instead of ssh tunneling x410 suports AF_VSOCK for the x socket, which hyper-v supports for performance as good as a domain socket on the same machine.
* Sparse & light - they only allocate resources from the host when needed, and release them back to the host when freed * Fast - it can boot a WSL distro from cold in < 2s * Transitional - these lightweight VMs are designed to run for up to days-weeks at a time
Full Hyper-V VMs aim to (generally) grab all the resources they can and keep hold of those resources as long as possible in case they're needed. Full VMs are designed to run for months-years at a time.
WSL's VMs are MUCH less impactful on the host - FWIW, I run 2-3 WSL distros at a time on my 4 year old 16GB Surface Pro 4 and don't even notice that they're running.
I imagine this will be addressed, but claims of lightweight seem exaggerated?
But even more on my mind is the impact on the windows host. Is it running as a guest under hyper v? What's the overhead?
This leads me to believe that display support is intended in the future. It's a work in progress. They've gone this far why would they stop at compute? Still, it's pretty awesome if you ask me.
I was a tad bummed when realizing what this actually was, but still very much impressed.
My experience is that Linux has significantly worse hardware support than Windows, particularly where newer hardware is concerned.
This is the first draft of the Microsoft Virtual GPU (vGPU) driver. The driver exposes a paravirtualized GPU to user mode applications running in a virtual machine on a Windows host. This enables hardware acceleration in environment such as WSL (Windows Subsystem for Linux) where the Linux virtual machine is able to share the GPU with the Windows host.
So this isn't actual "DirectX on Linux", just a driver for a virtual GPU exposed to WSL-guests to enable guests to directly use DirectX, more or less.
Probably most useful for Azure-based projects.
It is an extension of the capability of WSL, giving you that sweet convenience of the DirectX API with your existing ML project. Of course, this extension makes your project incompatible with desktop Linux once adopted.
It seems like Microsoft is continuing to see Linux as a production server target while positioning Windows to remain relevant as a workstation OS.
Ostensibly, this is a move to compete with Mac OS and not Linux.
Sure, nothing stops you from targeting DX12 directly in your application code, but why would you do that? At that point, you'd just target Windows since your users would have to be running it anyway.
It's an implementation detail -- they're exposing the Windows graphics driver to the Linux system with the most minimal amount of translation and overhead.
You could code directly to it in your Linux application code but it makes no sense to do that. You'd be literally writing a Linux application that can only run under Windows -- the smallest market ever proposed. Instead library/framework developers will add it as another target to improve performance in WSL for generic Linux applications.
Ah, but it does tie the developer to Windows ~ this DirectX module can only be used with WSL2, which only runs on Windows.
Unless you think developers would actually bother coding explicitly for the world smallest possible market (Linux inside of Windows).
This is the real and full D3D12 API, no imitations, pretender or reimplementation here… this is the real deal. libd3d12.so is compiled from the same source code as d3d12.dll on Windows but for a Linux target. It offers the same level of functionality and performance (minus virtualization overhead).
>> Linux when running in WSL.
>> minus virtualization overhead
refers to "para" in "paravirtualization." "Paravirtualization" is reducing the overhead of virtualization when the guest knows it's being virtualized.
You do not control windows, it controls you. You must adapt to it being in your life and the choices made by its designers, this is both not secure and insanity for a software engineer to allow as an ongoing situation.
Even dual booting windows will mean its updater will overwrite the partitioning table of your drive and hide any other OS you have installed, and as I understand this people only do it out of need and not out of want.
I think you're giving MS too much credit here. Apple has one UI, and Windows has one UI (2 if you count windows in tablet mode). Linux has, what? 30?
Quoth the Torvalds:
"I still wish we were better at having a standardize desktop that goes across all the distributions… It’s not a kernel issue. It’s more of a personal annoyance how the fragmentation of the different vendors have, I think, held the desktop back a bit."
Only recently did Ubuntu stop trying to push Unity and accepted Gnome, thereby reducing fragmentation by 1.
https://itsfoss.com/desktop-linux-torvalds/
This is exactly what we don't want. There's already plenty of effort wasted on GBM vs. EGLStream so introducing a third API would just lower quality even more.
And the outcome of implementing this seems to ensure it will stay that way.
That's a strange complaint. Would you be happier if this was merged into the mainline kernel instead?
If I understand it correctly, it just means that GPU-accelerated code that previously only ran on "proper" Linux now can run on WSL too, being powered by DirectX behind the scenes.
What's bad about that?
> DxCore & D3D12 on Linux
Projecting a WDDM compatible abstraction for the GPU inside of Linux allowed us to recompile and bring our premiere graphics API to Linux when running in WSL.
This is the real and full D3D12 API, no imitations, pretender or reimplementation here… this is the real deal. l
> libd3d12.so and libdxcore.so are closed source, pre-compiled user mode binaries that ship as part of Windows. These binaries are compatible with glibc based distros and are automatically mounted under /usr/lib/wsl/lib and made visible to the loader. In other words, these APIs work right out of the box without the need to install additional packages or tweak the distro’s configuration. Support is currently limited to glibc based distros such as Ubuntu, Debian, Fedora, Centos, SUSE, etc…
So you can use it with MS blessed distros and only when running under WSL2. You can't use it without WSL2.
https://lkml.org/lkml/2020/5/19/1139
And backing that up, they ported Mesa to get OpenGL and OpenCL in the Linux guest and are working on a Vulkan port as well.
Running Windows VMs as kernels for Cygwin environments felt less weird than most modern container architectures.
The goal, rather, is to take all the usual (for the Linux side) GPU stuff and give it access to the Windows host's GPU, when running on WSL.
That is, they are already working on getting OpenGL/Vulkan/etc. to run on top of this: https://www.collabora.com/news-and-blog/news-and-events/intr...
Rather than "get people to write code that runs on WSL2 but not regular Linux," the goal is "get code that runs on regular Linux but not WSL to run on WSL2."
I don't see why it would be, given the circumstances.
Perhaps their interests will shift in the future, as they clearly have in the past. But the things they build now are not from the "extend" phase of an EEE arc. At worst they are in the "embrace" phase.
If MS make it as easy to run desktop apps on Windows as it is on Linux, then the question of why even run a dedicated Linux machine if you can do everything on Windows? becomes relevant.
I suppose clipboard integration already works? And drag n' drop?
Once that mindset is heavily entrenched (e.g. in Enterprises), then the extend phase can begin.
I don't think MS are as scared as they were previously when Linux started to dominate the server landscape, and swathes of new developers (especially web backend) moved to a java, ROR and Python (or LAMP), where MS were absolutely nowhere in the stack.
MS is not in the business of paying developers to give away nice things for free.
Just like all positive sum games, open source is in everyone's interests, including Microsoft's. Egoistic alturism: serving your own interests [through] serving everyone else's interests.
All the people here railing on how Microsoft is destroying open source clearly don't understand the very premise of the thing that they think they are defending.
Unfortunately, the developers don't call the shots in there. And the ones calling the shots don't give a single damn about these developers.
Microsoft is back to it's old shenanigans after a big PR blitz.
And your evidence being what exactly? Yes, they’ve done EEE before, but there’s no evidence that they are doing it now. It’s been two decades.
While the actual meat of directx will still be proprietary and windows-exclusive where they call the shots and make the money.
How do you know they don’t plan to port DirectX to Linux? They’re currently attempting to upstream their changes, so why wouldn’t they port DX itself? They also don’t have to open source it (although I wish they would); they could just release a binary blob that you download and install.
Also, how do you expect Microsoft to kill Linux? To me, it seems that people who willingly choose to use Linux over Windows are generally the kinds of people who won’t be persuaded to move back. Microsoft can’t kill Linux.
Have you ever looked past your preconceived biases and considered that maybe Microsoft has actually had a change in culture over two decades? I don’t think so, because the people who scream “EEE” are the kinds of people who’ll never be swayed in their opinion.
Which is exactly what I was expecting since day one when WSL was announced. WSL is going to slowly kill Linux, or to be more accurate, it will kill any Linux (be it on servers or desktops) not Microsoft branded and distributed. Forks on smaller embedded systems will resist for a while, until one day MS decides to port some killer technology de facto embracing them as well.
What really stops me from using Linux on the Desktop is compatibility issues with my laptop(s), reliable sleep/wake and rendering issues with High DPI display.
Microsoft released an MS Teams client for Linux which astonished me. So, I am giving them the benefit of the doubt for the time being.
The Windows protectionism that dominated the 2000's doesn't help Microsoft sell more product anymore.
It's not so much that Microsoft has changed but that the world has changed and Microsoft is changing with it.
Still, thanks, I guess.
It’s going after macbook’s position as the default developer laptop and Apple deserves what’s coming to them IMO, for neglecting it for so long.
Perhaps in your country/field. Certainly not in mine...
DirectX does run on regular Linux, courtesy of Wine/Proton. It's just a matter of reimplementing the userspace .so interface that MS will provide for access to this facility, so that it hooks into that support instead.
They and the clones won the war with IBM in the 80s and 90s because IBM was worried about things that didn't matter: they wasted time with things like an operating system CP/X86 that ran DOS and 3270 emulators to their mainframes on top of a GUI system called mermaid (3270 PC) while Microsoft was writing Windows 3. Oh yeah did I mention some configurations cost $20,000 each? some configurations cost $20,000 each.
Because IBM wanted to secure its AS/400 minicomputers, mainframe, and microcomputer line and make them interoperable, as if a PC user sitting in their den would be connecting with $500,000 or so of other IBM machines, everyone else ran circles around IBM. They kept up the "whole kitchen sink" system way way past its due date.
This also happened to DEC, which responded way too late in the game to be relevant, SUN, SGI, and Wang. And it may eventually take down Oracle.
So they're intentionally taking an uncommitted, decoupled approach. I'm paying Microsoft as a result. I pay GitHub and Azure bills every month and run exclusively Linux.
They can't sway around as a monopoly like they could in 2000, once Ballmer left things changed rapidly. It's not going back. They have effectively zero cloud software (database/operating system etc), effectively zero mobile presence, and most people don't really like Windows.
In a tightly coupled stack you're only as strong as your weakest link, and MS has a bunch. Their biggest risk now is to HP or GM themselves; basically gobble up a bunch of things, blend it into indifferentiable blandness and collapse while waving a giant sceptre labelled "greatest company of 30 years ago" (HP bought DEC, Cray, Tandem, Apollo, Convex, 3Com, Phoenix, Palm and SGI and did effectively nothing with them beyond slapping an HP logo on their final pre acquisition product line and then just rode it out without any followup. HP knowing only how to fumble the ball every time for 20 years is why big business switched to Linux. HP took all the alternatives behind the barn one by one, cut them a fat check and then shot them. Simply crazy)
So yeah, it's a different game now.
Oracle is basically a department of the NSA/CIA/FBI. Sales to companies are just cover for the real game. It will never go away.
[1]: https://github.com/NVIDIA/nvidia-docker
[1] https://devblogs.microsoft.com/directx/directx-heart-linux/
[0] I use this word specifically because it's a martial art about using an opponent's strength against themselves. MS's use of Chrome as the basis for their new browser is another example of this.
You won't be able to take this dxgkrnl driver and load it into a Linux workstation and get DirectX.
And also, someone else responded to you what's the point of running Linux if Windows can do everything Linux can.
I would assume that most full time Linux users are using it out of at least a tiny bit of ideological motivation. And if not ideological then habitual, having had the power and granular transparency of Linux for so many years that Windows would never be an alternative.
Running Windows has never been an option for me, since Microsoft went down the "activating Windows" path many, many years ago.
Why?
This is not DirectX on the Linux desktop, it is DirectX for WSL. This means that code written for this will NOT function on desktop Linux.
Additionally, support for DirectX for WSL is support not spent on Vulkan and OpenGL.
Like CUDA, I assume that most applications would not be coded to this API directly. Microsoft already mentioned OpenCL and OpenGL.
This is to hardware accelerate Linux applications and not to create WSL-specific Linux apps. I can't imagine there's a big market for Linux GUI apps that would only run on Windows.
Not a fan but also not a graphics programmer.
> This is a driver that connects a binary blob interface in the Windows kernel drivers to a binary blob that you run inside a Linux guest. It's a binary transport between two binary pieces. [...] I can see why it might be nice to have this upstream, but I don't forsee any other Linux distributor ever enabling it or having to ship it, it's purely a WSL2 pipe.
1. https://lkml.org/lkml/2020/5/19/1288
That being said I don't see why distros like Ubuntu that allow non free code wouldn't ship with it out of the box. They already support Nvidia binary drivers anyway, so why wouldn't they support a transport bridge that'd let people use them on WSL too?
Also, he notices that it's intended to be more than a bridge in the future, as they're already working on integrating it with the presentation layer on Linux.
On top of that, why would canonical care what hypervisor you're running? There's so many other cases where they haven't gone "eww.. proprietary", why would they start now.
This is a way to lock developers who want to run Linux code for AI/ML onto their windows machines and to keep them from switching their desktop OS to Linux.
Edit: from the kernel mailing list
> There is a single usecase for this: WSL2 developer who wants to run machine learning on his GPU. The developer is working on his laptop, which is running Windows and that laptop has a single GPU that Windows is using
Meanwhile today, Nvidia and AMD GPUs don't support SR-IOV. So that's a non-starter. It's also not clear that it the right model for GPUs either, as they have their own MMUs already.
> Kvm setups can already pass through gpus just fine.
But you need to dedicate that GPU to the guest and the host can't use it anymore.
>> > There is a single usecase for this: WSL2 developer who wants to run machine learning on his GPU. The developer is working on his laptop, which is running Windows and that laptop has a single GPU that Windows is using
From the current patch set, because they haven't hooked up the swap chains yet (as they've said that they're doing) which would allow full graphics support in the guest.
Microsoft can ship it however they please though within bounds of licenses.
And why do you assume it only helps WSL? Doesn't this help all hyper-v setups get access to GPUs, including azure?
FTR, I'm not arguing whether or not this should be upstreamed. I just see where the other poster could be coming from. I could be wrong on my take and if so, someone please correct me.
Would y'all be so worked up about VMWare upstreaming their virtual GPU bridge?
And yet here's where their paravirtualized GPU driver lives https://github.com/torvalds/linux/tree/master/drivers/gpu/dr...
It feels like they're doing everything right and everything we've asked for, for decades, and being shunned for it. What's the mechanism here for Linux winning and Microsoft being allowed into the community and what should they be doing different?
What exactly? The only performant way to expose GPUs to guests while still allowing them to be used by the host (or multiple guests) for GPUs that don't have hardware support for partitioning themselves (ie. pretty much all discrete GPUs) is to replicate the ioctl layer from the host up into the guest. Any other solution is a non starter if you care about getting the perf you'd expect from the paravirtualized GPU.
https://lkml.org/lkml/2020/5/19/960 https://lkml.org/lkml/2020/5/19/1288
OpenCL, DirectX and CUDA all seem to be available for GPU acceleration on WSL.
One other exciting thing is GPU accelerated GUI Linux apps on Windows. You may not need an X server anymore. I'm looking forward to using i3 on Windows.
https://lkml.org/lkml/2020/5/19/1139
"This is a driver that connects a binary blob interface in the Windows kernel drivers to a binary blob that you run inside a Linux guest."
https://lkml.org/lkml/2020/5/19/1288
This isn't DirectX on Linux. This is DirectX API access for WSL exclusively. It will lock ML projects into Windows/WSL for the price of access to the DirectX API and at the expense of development of other projects like Vulkan for native Linux.
The whole point looks to take your CUDA code that would run just fine on Linux without a hypervisor, and run it just as well in WSL2.
ie. this is at worst still the embrace stage, IMO.
At the moment, that library only works under WSL, but I guess the DXVK folks can connect their implementation too.
https://lkml.org/lkml/2020/5/19/1139
And backing that up, they ported Mesa to get OpenGL and OpenCL in the Linux guest and are working on a Vulkan port as well.
Someday? You can already game on Linux.
Typical..
"Windows could be required to use some Linux apps (Embrace, extend, extinguish)" -> https://news.ycombinator.com/item?id=23238790
The most important is to look at the following page to better understand the implications:
https://devblogs.microsoft.com/directx/directx-heart-linux/
Here is how I would sum it up:
* They create a new kernel file descriptor: /dev/dxg
* With this, on the Linux of WSL, it is like a direct "pipe" to a Windows host graphical stack.
* So it means that they can put code in Linux application code that will use Windows proprietary graphical stacks, like DirectX through /dev/dxg.
* And so, clearly, this Linux app will not work inside a normal Linux computer that is not a "guest" of Windows.
Now, you can see the "Embrace" and the "Extend"?
And to well understand the article, the following things have to be stressed: This is not just something to render a Window or something like that, it is a special "passthrough" api to all the things that are provided by Windows GPU stack/drivers.
For example, they give the example of Cuda compute API, but also, they kind of "built" DirectX sdk itself for Linux (debian, ubuntu, ...), but still the proprietary closed blob that "apps" are expected to use, but that will rely on /dev/dxg.
Also, if you want to use anything OpenGL on the Linux WSL, they will ensure that it is translated to DX on Linux side, before going through the same special DirectX api. The good thing would probably have been to do the opposite side, ensure that Windows has the proper OpenGL support to be able to pass gpu acceleration to the Windows host.
There is a pre-existing closed-source NVIDIA Vulkan+OpenGL+CUDA library for Linux apps that speaks EGLStreams to /dev/nvidia0.
There is an announcement that the closed-source NVIDIA library might start speaking WDDM to /dev/dxg.
There is a pre-existing open-source Mesa DirectX 12 (+Vulkan +OpenGL + ...) library for Linux apps that speak GBM to /dev/dri (and some support for speaking EGLStreams to /dev/nvidia0 too).
There is a pull request to implement /dev/dxg in the kernel as a Hyper-V pipe for WSL2's use.
There are lots of interaction points for alternate implementations of various pieces of the stack. For example, in the future it might ultimately be possible for an alternate /dev/dxg implementation to wrap the normal DRM API, or for NVIDIA's binary driver to reimplement /dev/dxg on real hardware.
Meanwhile MS wants to cast widest possible net, so it needs to enable more and more workflows on WSL2 today, not tomorrow.
EDIT: Looking through an old Intel whitepaper, it looks like the system firmware at most has to reserve some extra config space for SR-IOV devices when enumerating the PFs, so that the OS can enumerate VFs after creating them. But Linux includes an option to re-allocate this stuff if the BIOS doesn't reserve space, so this apparently isn't a hard requirement.
Intel is able to get away with it for integrated GPUs because they can codesign their system level IO-MMU with the GPU MMU and it doesn't have it's own VRAM bank. I'd bet dollars to donuts that Intel's new Xe discrete GPU doesn't support SR-IOV even though the integrated versions of the same core will.
The same with Nvidia's A100.