OP here. this is a cursed project lol, but i wanted to see: What happens if you replace the OS scheduler with an LLM?
With Groq speed (Llama 3 @ 800t/s), inference is finally fast enough to be in the system loop.
i built this TUI to monitor my process tree. instead of just showing CPU %, it checks the context (parent process, disk I/O) to decide if a process is compiling code or bloatware. It roasts, throttles, or kills based on that.
Its my experiment in "Intelligent Kernels" how they would be. i used Delta Caching to keep overhead low.
But seriously, it does really bug me on principle that DropBox should use over half a GB simply because it uses Chromium, even when nothing is visible.
For me it's LSP servers taking 2 gigs of RAM. With Antigravity, Google managed to go beyond this, it is totally unusable for me (but other VScode clones work fine, apart from the 2 Go LSP servers).
Now we need processes to gain awareness of the process manager and integrate an LLM into each process to argue with the process manager why it should let them live.
oh absolutely. burning a coal plant to decide if i should close discord is peak 2025 energy.
strictly speaking, using the local model (Ollama) is 'free' in terms of watts since my laptop is on anyway, but yeah, if the inefficiency is the art, I'm the artist.
An interesting thought experiment - a fully local, off-grid, off-network LLM device. Solar or wind or what have you. I suppose the Mac Studio route is a good option here, I think Apple make the most energy efficient high-memory options. Back of the napkin indicates it’s possible, just a high up front cost. Interesting to imagine a somewhat catastrophe-resilient LLM device…
Macs would be the most power efficient with faster memory but an AI Max 395+ based system would probably be the most cost efficient right now. A Framework Desktop with 128GB of shared RAM only pulls 400W (and could be underclocked) and is cheaper by enough that you could buy it plus 400W of solar panels and a decently large battery for less than a Mac Studio with 128GB of RAM. Unfortunately the power efficiency win is more expensive than just buying more power generation and storage ability.
You're underselling this as a process manager, it could also be a productivity tool with some prompt changes; Determine procrastination apps: games, non-professional chat, video streaming and kill it.
28 comments
[ 4.2 ms ] story [ 66.8 ms ] threadWith Groq speed (Llama 3 @ 800t/s), inference is finally fast enough to be in the system loop.
i built this TUI to monitor my process tree. instead of just showing CPU %, it checks the context (parent process, disk I/O) to decide if a process is compiling code or bloatware. It roasts, throttles, or kills based on that.
Its my experiment in "Intelligent Kernels" how they would be. i used Delta Caching to keep overhead low.
I wouldn't call it replacing the scheduler though - more that you've made a scheduler manager.
But seriously, it does really bug me on principle that DropBox should use over half a GB simply because it uses Chromium, even when nothing is visible.
Scheduler Manager is definitely the more accurate term. Im just the middleman between the chaos and the kernel.