HN Explorer

Show HN: Blotter – An interactive, never ending music video (twitch.tv)

166 points by ck019 ↗ HN
One day I was listening to a playlist and wished there could be some cool visuals to go along with it.

Blotter is a proof of concept I hacked together that does a bit of audio recognition combined with a few generative AI models (both text and img) to create visuals that are relevant to the song.

The video stream is generated in real time at 24fps - you can try it yourself by requesting visuals in the Twitch chat using the "!v" command!

Right now it's mostly a fun hack project, but I am tinkering with new model architectures for higher fidelity video as well as an interactive tool so people can make videos with their own audio files.

I'd love to hear any feedback or suggestions, thanks!

40 comments

[ 0.27 ms ] story [ 93.6 ms ] thread
samradelie
You made the ultimate VJ party tool.
tanseydavid
AI tool killed the VJ-stars' gig (with a nod to "The Buggles")
ck019
Woo! Thanks for checking it out
esalman
When I opened it someone requested visuals of Biden and Putin kissing.

Slippery slope I say this.

echelon
> Slippery slope I say this.

Or humans need to chill out, realize we're a bunch of primates on a space rock, and stop being so offended and afraid.

So what if we use politicians, world leaders, and celebs? This is no different than photoshop.

You know what's funny?

https://www.youtube.com/watch?v=IkaAZE_UGMo

https://www.youtube.com/shorts/ivhUH68KWis

Or something along those lines:

https://www.youtube.com/watch?v=03vmbKGW1iI

This is even funnier to me because I have no idea what any of this means. Kids love this stuff, though, and they're wildly creative.

We're just getting started with the creative Cambrian explosion. We have yet to hit the inflection point.

esalman
Do you think twich should have moderation?
echelon
I think Twitch should be a protocol.
seth_
Would be neat to see the distribution of time that people stare at this... I imagine some will stare for hours
ck019
I've definitely fallen into that trap before lol
brianhama
Are you going to release the code?
ck019
Yeah! It'd be cool to see what others build on top of this.

Gonna add cleaning things up a bit and writing some documentation to my to-do list

mdrzn
Appreciate that, I would love to tinker and play with it and see what comes out.

Always loved things that go in time with music

samradelie
Very keen on this. Expressed interest on your web form
lovemenot
The music fits this project well. Is it always a lofi channel?
ck019
Thanks for checking it out! Yeah right now it's all lofi - was thinking about making a few different streams for different genres though
lovemenot
Cool, but I will stick to this stream, thanks. Loving it. Stream name is inspired by little squares of cardboard?
ck019
Haha indeed it is
pluijzer
Really nicely done, the visuals really react well to the music. I am myself not a big fan of lofi but can imagine having this on as a nice background with some ambient music.
candiddevmike
I can't do generated images, especially like in this video. Makes my head hurt or makes me nauseous, either way I can't imagine who would really watch nothing but unfiltered image gen like this. I think you'd go insane trying to make sense of it.
ck019
Totally get that!

Thanks for taking the time to check it out either way!

erur
A neverending stream of dynamic content based on audience interaction. It's way more entertaining than I anticipated. This should also be fairly easy to monetize - either by having people donate for interaction or by including ads.

This feels like a solid glimpse of the truly new things that we'll likely see take shape soon.

ck019
Definitely wild to see how far the tech has come in even the past few months! Thanks for checking out the stream!
xtagon
Fun! Mind sharing which ML models you used?
ck019
Yeah! It's actually a fair bit of custom code tying together parts of multiple models.

Hoping to clean it up a bit and share an e2e repo soon

djbusby
I'd be happy to even see the un-clean repo.
elisbce
This is so fun, although it is super grainy. Is it because you cannot run the generation at higher res in time?
ck019
Yep that's right! This project was definitely an exercise in balancing visual fidelity and inference speed. I have a couple of ideas to increase realism though that I'm excited to dig into and hopefully have live soon.

Thanks for checking out the stream!

frog69
what kind of machine are you running this on?
ck019
g4dn.2xlarge

Thanks for checking out the stream!

numlock86
It's interesting to see what people come up with in the chat, other than the typical Shrek meme prompts and the like. How do people even come to think of stuff like "demon core party prank"? Wild.
ck019
It was definitely as entertaining watching chat as it was watching the visuals!

That being said, one of my biggest takeaways from this whole experience was I could've just trained a Shrek generator and it would've covered like 90% of use cases

tudorw
nice idea, weird visuals, I'd prompt it to stay in the abstract zone maybe... twitch now has What The Robot Saw, WatchMeForever, AI Jesus, and my channel m88t where I run an AI Nursery...
neom
Cool that it knows other languages.
KomoD
It's pretty fun
ck019
Love to hear it! Thanks for checking out the stream
uni_baconcat
A really interesting project. I would try to leave this playing tonight as a screensaver.
ck019
Thanks for checking out the stream!
maxmin360
I am currently learning how to mix house music. Would totally use something like this to keep my audience visually engaged.
ggrelet
That's amazing, how can I subscribe and follow your next improvements on this?