As a professional arranger and composer, I can confirm that this is indeed unconvincing and disappointing (although I will admit to being mildly impressed by some of the generated orchestra textures, but only because this is coming from a computer program, not because they are actually good).
It is my firm belief that in maximum 10 years we will have number one hit songs completely produced end-to-end by a neural network - instrumental, lyrics, vocals, mixing.
I mean we're not far off with Vocaloid. Ever seen a Hatsune Miku concert [0]? All that needs to be met is "generate lyrics and song by A.I" and I've already heard a number of A.I generated that could work well enough as backgrounds for a vocal-heavy track [1]. The hardest part would be training a neural network to generate Miku's vocals and maybe creating the lyrics for the song.
> Our results present abilities that are, as far as we know, unheard of.
Look, I gotta say, I'm pretty disappointed with the ridiculous level of salesmanship that authors now feel is necessary to get a paper into NIPS. I can only hope the reviewers might ask the researchers to tone back the crazy hype pitch, but I know better than to expect that.
Presenting things that were previously unheard of... well, that would be what scientific papers are for, aren't they?
> Presenting things that were previously unheard of... well, that would be what scientific papers are for, aren't they?
Not really. Just replicating existing findings, summarizing knowledge that would otherwise be spread out across multiple papers, articulating an opinion, or just applying something another paper did in a similar fashion ("I did the same but with blues music!") is a perfectly valid endeavor.
You should however state what your paper contains somewhere at the beginning, which is what they did there - albeit in a "flashy" manner.
That indiana jones theme whistling to organ was hilariously awful -> https://youtu.be/vdxCqNWTpUs?t=124 . Though I guess you can blame a lot of that on how bad the whistling was in the first place.
But I would totally play with this if given the chance. I don't foresee this tech replacing human composition for pop music, as one comment here expects, but it will be a tool for musicians in the same way that auto-tune became it's own sort of sound/style. I definitely can see it being useful for prototyping arrangements for instruments that you can't play (or don't keep around).
Funny, that was the most exciting example for me. A whistle2orchestra app would be great fun. Agree the input sample should have at least had the breath scrubbed out.
I mean... this mostly just feels like the originals were fed through some sort of vocoder... None of this is impressive at all to me. Translate orchestral output to hip hop or something and then I'll be impressed. At this point, you've fed one song in and output some kind of grainy sounding similar sound that is basically just triggering notes in a vocoder...
Yes it's definitely working on the note, envelope and timbre level as opposed to the production, mastering, soundscape, etc. level. But I think it's ok as a stepping stone. Something like this could still work as a backing layer for the original.
It's convincing in that they labeled each style "as such", but it's not convincing in a musical sense. Music is playful and joyous and inspirational, these clips are academic exercises. Let's see what can be done with the tech in years to come, but I won't put down the virtuosi for this anytime soon.
Music has meaning. True, it's a kind of meaning that we cannot put into words, but meaning nonetheless. These exercises do not appreciate this fact. (It's like taking a poem and generating another poem of a different author based on phonemes alone).
> It's like taking a poem and generating another poem of a different author based on phonemes alone).
There is already a school of poetry that does this. For the LANGUAGE poets, meaning is often secondary, what is really of interest is the permutations of existing material. As an example, see [1] where Charles Bernstein creates an English poem by choosing words that sound similar (but don't mean anything similar) to the German words in Heinrich Heine’s classic poem "Lorelei".
Interesting to consider its apparent strengths vs weaknesses, and what that may mean about each musical style (or at least the training sets). It seems to completely suck at generating listenable Beethoven, and its Bach is better but still not great, but this system creates quite passable Mozart.
It's not "convincing" and there is no sense of "style change" at all - it's very clearly just awkwardly re-orchestrating the melody with some timbral changes. The tonality is often entirely lost (the Rihanna to Mozart example is the most egregious).
I really hate that we have to criticize these awful articles and press releases all the time, because this work is actually really cool, but because the researchers and article writers constantly go overboard with their claims and rhetoric, the level-headed people have to come along and say: "look, this is cool, but it's nowhere near the claims you're making". I'm really sick of this happening with every DeepMind/FAIR/Microsoft/IBM result that gets published. It's so tiring.
Conflicted between being all for a company like Facebook throwing their money at AI vanity projects and that these researchers are working on AI vanity projects rather than something else.
27 comments
[ 200 ms ] story [ 863 ms ] threadYeah, it's been a sci fi trope for decades, but it's coming.
[0] https://www.youtube.com/watch?v=dhYaX01NOfA
[1] Just one example: https://www.youtube.com/watch?v=LSHZ_b05W7o
Look, I gotta say, I'm pretty disappointed with the ridiculous level of salesmanship that authors now feel is necessary to get a paper into NIPS. I can only hope the reviewers might ask the researchers to tone back the crazy hype pitch, but I know better than to expect that.
Presenting things that were previously unheard of... well, that would be what scientific papers are for, aren't they?
Not really. Just replicating existing findings, summarizing knowledge that would otherwise be spread out across multiple papers, articulating an opinion, or just applying something another paper did in a similar fashion ("I did the same but with blues music!") is a perfectly valid endeavor.
You should however state what your paper contains somewhere at the beginning, which is what they did there - albeit in a "flashy" manner.
But I would totally play with this if given the chance. I don't foresee this tech replacing human composition for pop music, as one comment here expects, but it will be a tool for musicians in the same way that auto-tune became it's own sort of sound/style. I definitely can see it being useful for prototyping arrangements for instruments that you can't play (or don't keep around).
There is already a school of poetry that does this. For the LANGUAGE poets, meaning is often secondary, what is really of interest is the permutations of existing material. As an example, see [1] where Charles Bernstein creates an English poem by choosing words that sound similar (but don't mean anything similar) to the German words in Heinrich Heine’s classic poem "Lorelei".
[1] https://jacket2.org/commentary/lorelei-heine-translations-sh...
I really hate that we have to criticize these awful articles and press releases all the time, because this work is actually really cool, but because the researchers and article writers constantly go overboard with their claims and rhetoric, the level-headed people have to come along and say: "look, this is cool, but it's nowhere near the claims you're making". I'm really sick of this happening with every DeepMind/FAIR/Microsoft/IBM result that gets published. It's so tiring.
There's a solution to this: stop linking to science journalism pieces. Instead, we should link directly to the paper.
Hopefully the patents don't get walled off.