HN Explorer

Ask HN: What's the status of AI-generated music?

7 points by raincole ↗ HN
Since Stable Diffusion and ChatGPT got popular, I'd like to ask what their equivalant are in the music field.

I understand that composing involves a lot of nuances and it's not just to put music notes down and call it a day. But it's still a little hard to believe that it's much more difficult to make an AI that, say, can compose music tracks like https://www.youtube.com/watch?v=OSPkn-iHPWA (Pokemon Sword's Title Screen), than to make ChatGPT.

Is Riffusion(https://www.riffusion.com/) the cutting edge tech? Or am I missing something?

5 comments

[ 4.0 ms ] story [ 20.9 ms ] thread
ofalkaed
AI is not very good at music yet, it can do things like decent imitations of composers like Bach who have a massive catalog for it to work off of and wrote under a fairly rigid theory but that is about it. Pop music is largely emotion based and does many nonsensical things (from a theory standpoint) based on emotion, AI is not good with that. The more rigorous types of music are far too fuzzy, every "rule" has an infinite number of exceptions which AI is not very good with either.

Played with riffusion some, not at all impressed by it. It is mimicking without understanding and while it occasionally did something interesting it has no real comprehension of time scales over short loops, no understanding of larger structures and fails completely on the prompt if it is something it can not easily research. This is about where AI generated music has been for a decade now and can not seem to push past. Part of this is probably because AI was integrated into composition a good long while ago, composers tend to treat it more like an instrument or a filter than something which writes music and most of the work with AI in music is towards those ends, not towards getting AI good at composition.

CrypticShift
MusicLM [1] is the most impressive demo I've seen yet. The problem is we only have access to selected examples (=demo) so we don't know how it performs "on average".

IMO, for music, the limiting factor today is more copyright (=datasets) than models per se.

[1] https://google-research.github.io/seanet/musiclm/examples/

ofalkaed
The only thing better about these than any other I have heard is that the synthesis quality is better. It just sticks cliches into new contexts and like riffusion and the rest probably has no ability for the longer structures and developing the piece over time in an effective fashion, hence the short samples.

If copyright was the issue than we should have AIs that can effectively do older styles of music which have a great deal of public domain music to work off of, but we don't.

CrypticShift
Yeah, the samples are short and "cliche", I agree. Impressive is subjective : creating that from text is already impressive for me.

Also, when I said copyright is limiting, I was thinking about generating "short" and "cliche" emulations of any style of music or artist (the way you do it in early stable diffusion for images)

I might be wrong, but I personally don't expect "innovative developing compositions" anytime soon.

notahacker
You can make a few lines of toy script that 'compose' a MIDI track like your example or other chiptunes, and play it with a digital instrument

I think that's one of the big issues with relative lack of emphasis AI generated music. Not only is the hard stuff relatively hard, the easy stuff that's generated to conform to regular Western melody/harmony norms played on a regular digital instrument is so easy it doesn't need NNs.

Musicians' copyright lawyers have also been extremely successful at demanding writing credits for 'inspiration' compared with other creative industries.