Look pretty. Can we load alternative stem separators like Spleeter, MDX-Net, and RoFormer implementations? I'd like to be able to AB them for different stems types.
Adding this to my homelab, thanks for the web browser option. How does your son use the backing tracks? Does he mute bass/drum, or play along with them?
Hello there, It depends actually, for what I hear from time to time when he is practicing, he sometimes plays with the original drums on default value or he loops into a section at lower speed and volume a bit down when he wants to learn an specific part. and when he feel confident he just mutes the drum completely :)
I just want to say, kudos on your We Recommend section.
I was initially rolling my eyes at the "StemDeck... does not accept any money, sponsorship, or funding" line, here we go, another open source project that isn't thinking about practicalities... until I saw you were linking to others as pure recommendations. Just for the joy of what they do & how they've helped you and hoping they do the same for others.
The web used to have a lot more of that. It's a shame that doing so now often requires a disclaimer, and comes with the suspicion of being an influencer, or being done for SEO. And certainly many open source projects have done their part in corrupting the web too, accepting payment in return for SEO links on their pages. (Don't get me started on some of the things Mastodon accepted payment for...)
Thank you for bringing back that more hopeful, joyous part of the web and the music community.
That's actually the whole point for me. As i have no itention to monetize anything. Most people that are there are people that i Personaly know and that have a positive impact on my life or companies like empress/ Thoman that im a fanboy and their support has been amazing towards any product that i bought with them.
Rust could certainly handle much of the audio pipeline, but using Rust everywhere would not automatically make the application simpler or more reliable.
Other options may be available but for me FFmpeg provides mature, well-tested support for decoding, transcoding, resampling, mixing, and muxing across a wide range of formats. Web Audio handles synchronized multitrack playback, per-stem gain, mute and solo, metering, looping, speed control, and the browser-based mobile interface. Since the separation model already depends on Python and PyTorch, rewriting the surrounding audio stack in Rust would not remove the largest runtime dependency.
As for RUst itself its currently used for the Tauri desktop shell and process lifecycle. A native Rust audio engine may make sense later if it produces a measurable improvement in latency, memory use, or reliability, but replacing proven components purely for architectural consistency would add considerable complexity.
For the vibe coding part:
AI tools have assisted with development, and I am transparent about that. However, the architectural choices are deliberate, the code is reviewed, and the project has automated backend and browser testing. I would still welcome specific technical criticism or examples of places where the current design is causing real problems.
That's where my background kicks in, seasoned musician here with over 25+ year working on IT industry, so as i like to joke, im the maestro of the orchestra :)
On another words. i actually know how to cook , but to do it faster i need the assistances otherwise as a family man, I would never have the time to ship that and help my kid on a useful time.
Funny - Stage Tour (a Rock Band revival game) announced this week. There was a big chat in their Discord about AI, and stem separation was one of the use cases that came up.
As a side, it's a bummer how many similar names there are. I keep seeing stuff about Stream Deck too. Apparently it's a custom keyboard for videographers, but it gets confused with Valve's Steam Deck gaming hardware when I see it in my feed.
concern from my side as well at some point, But The name comes from audio “stems” and the multitrack “deck” interface, but I understand that it can be confused with Steam Deck or Stream Deck. There is no affiliation with Valve or Elgato, and I’ll take any legitimate trademark concern seriously. Naming things really is the hardest problem in software. :)
For overall reference:
"In audio production, a stem is a discrete or grouped collection of audio sources mixed together, usually by one person, to be dealt with downstream as one unit. A single stem may be delivered in mono, stereo, or in multiple tracks for surround sound."
There is a robust community around it but it is not very user friendly; most info is found in discord and a Google spreadsheet and custom patches/builds of uvr
My only interest is if there are any newer, cleaner algorithms/models. htdemucs (as well as everything else in UVR) are good but not great and leave a lot of artifacts. The acapellas it stamps are 'good enough' til you get to final mixdown and then the flaws stand out real heavy.
It still blows my mind that this is possible, as someone who spent a lot of time as a kid trying and failing to get “acapellas” by EQing or subtracting instrumentals from vocal versions haha
Superb. I am so glad you built this (and I couldn't care less if you used AI for 5% or 1000% of it). Thank you so much for sharing what is obviously useful work with the community.
Also, with so many references to Lisbon I take it you're likely Portuguese or living here, so sending some love from Porto!
A while back I ported htdemucs (the same thing this uses) to onnx/wasm/webgpu so you can do it in browser (though I only bothered with the 4-stem version): https://bakkot.github.io/demucs-js/
70 comments
[ 0.28 ms ] story [ 21.9 ms ] threadI was initially rolling my eyes at the "StemDeck... does not accept any money, sponsorship, or funding" line, here we go, another open source project that isn't thinking about practicalities... until I saw you were linking to others as pure recommendations. Just for the joy of what they do & how they've helped you and hoping they do the same for others.
The web used to have a lot more of that. It's a shame that doing so now often requires a disclaimer, and comes with the suspicion of being an influencer, or being done for SEO. And certainly many open source projects have done their part in corrupting the web too, accepting payment in return for SEO links on their pages. (Don't get me started on some of the things Mastodon accepted payment for...)
Thank you for bringing back that more hopeful, joyous part of the web and the music community.
Seems like a great way to have your project taken down
Other options may be available but for me FFmpeg provides mature, well-tested support for decoding, transcoding, resampling, mixing, and muxing across a wide range of formats. Web Audio handles synchronized multitrack playback, per-stem gain, mute and solo, metering, looping, speed control, and the browser-based mobile interface. Since the separation model already depends on Python and PyTorch, rewriting the surrounding audio stack in Rust would not remove the largest runtime dependency.
As for RUst itself its currently used for the Tauri desktop shell and process lifecycle. A native Rust audio engine may make sense later if it produces a measurable improvement in latency, memory use, or reliability, but replacing proven components purely for architectural consistency would add considerable complexity.
For the vibe coding part:
AI tools have assisted with development, and I am transparent about that. However, the architectural choices are deliberate, the code is reviewed, and the project has automated backend and browser testing. I would still welcome specific technical criticism or examples of places where the current design is causing real problems.
That's where my background kicks in, seasoned musician here with over 25+ year working on IT industry, so as i like to joke, im the maestro of the orchestra :)
On another words. i actually know how to cook , but to do it faster i need the assistances otherwise as a family man, I would never have the time to ship that and help my kid on a useful time.
As a side, it's a bummer how many similar names there are. I keep seeing stuff about Stream Deck too. Apparently it's a custom keyboard for videographers, but it gets confused with Valve's Steam Deck gaming hardware when I see it in my feed.
Audacity can also do this through the OpenVINO plugins and I've been happy with its results (https://github.com/intel/openvino-plugins-ai-audacity).
We really suck at naming things...
For overall reference:
"In audio production, a stem is a discrete or grouped collection of audio sources mixed together, usually by one person, to be dealt with downstream as one unit. A single stem may be delivered in mono, stereo, or in multiple tracks for surround sound."
How does it compare to nuo-stems ?
That’s because it uses mel_band_roformer and bs_roformer which are really great stem separation models (i haven’t heard anything better yet).
Their page about stem separation quality is a nice place to start if you want to dive into these types of models: https://docs.nuo-stems.com/docs/stems-separation-quality
I assume phones are perfectly capable to run such software?
Also, with so many references to Lisbon I take it you're likely Portuguese or living here, so sending some love from Porto!