> Built for laptops with soldered memory and no upgrade path. If you have an RTX card sitting there with 8GB of VRAM and you're getting swapped to SSD, this puts that VRAM to work.
Well, that does at least answer my immediate question about why I would ever swap from expensive RAM to really expensive RAM:) Feels niche, but when you want it it's a good idea.
In the olden days we called that a "RAM Disk" and it made our Atari STs go really fast!
On the old Amstrad PCWs that were everywhere at least in the UK in the mid 80s to mid 90s you could have up to 512kB of RAM, a fair chunk of which could be a RAM disk. This made compiling stuff in Turbo Pascal really fast too :-)
So can VRAM actually be used like regular RAM? E.g. if I have a 16GB module and my GPU has 16GB VRAM, could it be made so that my system reports 32GB RAM? What would be the implications of that?
Remember how 16GBs used to be an enterprise level database mainframe?
Well, GPUs also have stupid amounts of compute on them. I have to imagine that there is some kind of database format that's useful with GPU compute attached.
Since the data is already in VRAM, the GPU can sort, join, or otherwise manipulate data as needed.
Having 128 GB in my desktop, I can never, ever go back. It truly unlocks a whole different computing experience. I've only had one OOM in the last 5 years and it was in my own code where I had a bad memory leak. It's the only way to live
For me opening huge datasets, e.g. many gigabytes worth of profiling data, combined with other stuff running on the system, can end up pushing things to swap.
Is not popular in general, so yes. But also no - I don't use swap ever, if I have to go over the RAM (32GB being low, with 64GB the norm), might as well consider the system dead.
What about backpressure, how does it handle requirements for VRAM allocation when VRAM is used for swap space?
With X11 it's not that bad (buffers are pre-allocated), but with Wayland allocations are a lot more dynamic, so running low on VRAM can easily crash the whole desktop. I just had a few of such crashes with Hyprland+llama-server+KVM switching between computers without freeing VRAM.
I seriously looked at this as a way to improve the RAM situation in a QNAP 2U unit that I was having trouble sourcing RAM for. It's somewhat annoying that legit memory-over-PCIe is gated on PCIe5 and chipset support.
In the end I just had to bite the bullet and take a gamble on finding ECC DDR4 RAM that would work with the ancient AMD chipset...
This particular implementation seems to be running over too many layers to be particularly performant. Why not a custom block driver instead?
As much as I'm avoiding GenAI myself⁰ I think your reaction is what feels a bit weird. You wouldn't be sending a tip for simply prompting the LLM, but for having the original idea and verifying/testing the result. If you don't feel right donating for that, then don't. Seeing a “buy me a coffee” link is hardly onerous, and it isn't exactly in-your-face here (I didn't notice at all until your comment mentioned it).
--------
[0] I want to code, I like the nitty-gritty, and if I want to outsource I'd prefer to outsource to a human¹ than GenAI
[1] they might outsource to GenAI of course, that is their choice and as long as they properly verify the output before handing it on to me I shouldn't have to care
I remember this being a thing done a while back using linux's MTD/phram drivers - https://wiki.archlinux.org/title/Swap_on_video_RAM - not sure if that's still relevant though as I don't know how it'll interact with DRM and how it handles reserving some of the vram - the suggested limit using xorg.conf is probably pretty obsolete now.
That page also has a fuse filesystem implementation on top of opencl - https://github.com/Overv/vramfs - which may be more compatible.
For windows I saw something similar to this years ago. An experimental proof of concept driver that allows the creation of a ram drive from vram for NVIDIA cards. Sequential is fast as you'd expect, random has lots of room for improvement.
53 comments
[ 3.1 ms ] story [ 71.0 ms ] threadWell, that does at least answer my immediate question about why I would ever swap from expensive RAM to really expensive RAM:) Feels niche, but when you want it it's a good idea.
On the old Amstrad PCWs that were everywhere at least in the UK in the mid 80s to mid 90s you could have up to 512kB of RAM, a fair chunk of which could be a RAM disk. This made compiling stuff in Turbo Pascal really fast too :-)
Well, GPUs also have stupid amounts of compute on them. I have to imagine that there is some kind of database format that's useful with GPU compute attached.
Since the data is already in VRAM, the GPU can sort, join, or otherwise manipulate data as needed.
>Sequential throughput: ~1.3 GB/s
[on a RTX 3070 Laptop]
This RTX 3070 chip is on PCIe 4.0 x16 which should give 64GB/s. The 8GB of GDDR6 is 448GB/s.
Swapping to an NVMe drive would be twice as fast, but with higher latency.
Now if it could be dynamically used and vacated on other GPU workloads?
Is not popular in general, so yes. But also no - I don't use swap ever, if I have to go over the RAM (32GB being low, with 64GB the norm), might as well consider the system dead.
With X11 it's not that bad (buffers are pre-allocated), but with Wayland allocations are a lot more dynamic, so running low on VRAM can easily crash the whole desktop. I just had a few of such crashes with Hyprland+llama-server+KVM switching between computers without freeing VRAM.
In the end I just had to bite the bullet and take a gamble on finding ECC DDR4 RAM that would work with the ancient AMD chipset...
This particular implementation seems to be running over too many layers to be particularly performant. Why not a custom block driver instead?
--------
[0] I want to code, I like the nitty-gritty, and if I want to outsource I'd prefer to outsource to a human¹ than GenAI
[1] they might outsource to GenAI of course, that is their choice and as long as they properly verify the output before handing it on to me I shouldn't have to care
That page also has a fuse filesystem implementation on top of opencl - https://github.com/Overv/vramfs - which may be more compatible.
Man, that brings back memories.
>GpuRamDrive
>Create a virtual drive backed by GPU RAM.
https://github.com/prsyahmi/GpuRamDrive
Fork with AMD support:
https://github.com/brzz/GpuRamDrive/