Show HN: I trained an AI model on 120M+ songs from iTunes (maroofy.com)

753 points by subtech ↗ HN
Hey HN!

I just shipped a project I’ve been working on called Maroofy: https://maroofy.com

You can search for any song, and it’ll use the song’s audio to find other similar-sounding music.

Demo: https://twitter.com/subby_tech/status/1621293770779287554

How does it work?

I’ve indexed ~120M+ songs from the iTunes catalog with a custom AI audio model that I built for understanding music.

My model analyzes raw music audio as input and produces embedding vectors as output.

I then store the embedding vectors for all songs into a vector database, and use semantic search to find similar music!

Here are some examples you can try:

Fetish (Selena Gomez feat. Gucci Mane) — https://maroofy.com/songs/1563859943 The Medallion Calls (Pirates of the Caribbean) — https://maroofy.com/songs/1440649752

Hope you like it, and would love to hear any questions/feedback/comments! :D

438 comments

[ 3.8 ms ] story [ 339 ms ] thread
Nice idea. Not sure how good the recommendations are just yet, but I really wish there was a Spotify button on there.

Also if it's not a trade secret can you expand "My model analyzes raw music audio as input and produces embedding vectors as output." What kind of analysis are you doing? What is the criteria for picking recommendations? I always find this area of music analysis intersting.

It's fast.

But not sure if the song I input and the songs I get back are similar. Going to try them out further.

What things are they supposed to be similar at? beats? melody?

Really cool, quite surprising that this hasn't been done (well) before afaik.
This is the Spotify Radio feature. i.e. find a song and start a radio from that song/album/artist
(comment deleted)
I searched for "Hot Chip" and all the recommendations were other Hot Chip songs... is that normal? I was expecting different artists, and honestly not expecting any songs by the same artist.
Pop in a specific Hot Chip song and it'll work better I reckon
Wow, these songs were impressively similar to my queries. I would love for a interpolation playlist feature, where I put in 2 different songs (from say, classic rock and EDM), and I could get 10-20 songs that slowly change from the start song to the end song.
Yes, this: can you get any interesting insights by playing with the embedding vectors? What happens if you add embeddings together? Weighted average of multiple tracks? Follow the average vector for an artist's work over time?
100% This is definitely worth exploring, and I'm currently trying to figure out the appropriate front-end UI/UX to expose this functionality for users.
Very cool, just tried it out and seems to work. Great job
What vector database do you use? Did you run into any scalability challenges there?
100% ran into a ton of scalability challenges lol. Maybe I should write a blog post about it sometime.

But for now, ended up using plain old FAISS.

Seems like your server is down or overloaded
Interesting idea. I think it needs some finetuning to find songs that are similar but not covers. For well-covered songs like Bohemian Rhapsody, this is more like a cover-finder.

Also this song, Shangri-La Is Calling, is bugging out: https://maroofy.com/songs/1632142336

"similar-sounding music" is indeed a good way to put it. 120 millions ! This is awesome! Reminds me of https://cyanite.ai/ (Search by audio). I will have to give it more time, but so far, I like your results better. Well done!
Fun, but unfortunately it is pretty off for me compared to Spotify and Apple Music. I wouldn't listen to any of the recs personally. I do appreciate how out there the recommendations are though! They're just amateurish for my tastes.
Curious, did you just get the album/song lists from iTunes and get the corresponding raw music audio from other sources or does iTunes really allow one user account to download the raw audio of 120M songs?
How did you download that many songs? Wouldn't that be something on the order of 360 terabytes?
I guess he is working with the 30s low rez previews, I know you can download them with the Spotify API. Apple Music should be similar.
Wouldn't this still be about 36 terabytes of data?
Meh, you can store that on a single d3en.2xlarge for $780.
You would only need to store the embeddings and not the file audio itself.
Could you elaborate?
Also curious about this... where did you get the dataset?
How did you get the 120MM songs?
I tried it, I think the vibe match looks great, but the match lacks the culture context. It would be great to use song's meta data and lyrics to to improve that
Very cool. Need a get a playlist from this. Seems like Spotify for us Apple Music peasants
That is a great idea! Measuring distance between embeddings has always been a cool concept (ex. If I have a vector that represents the word "king" and from it subtract the vector that represents "man" then add the vector that represents "woman" it will approximately equal the vector for "queen") and it's awesome to see the same concept applied to music.

Most other services try to find matches by seeing what other songs the people who like the searched song like, and finding trends amongst those. This site finds songs with similar sounds and rhythms by looking in the vector space. Awesome! Congrats on the finished site!

Can you share more details about the "custom AI audio model" used to generate the embeddings?

Curious what sort of metrics are taken into account, and if any supervision is provided.

Looks like a cool project. As long as it is free and not being monetized, then it might keep Apple from unleashing their lawyers.

Apple might as well buy your project for iTunes since they are extremely behind in anything AI.

This is very interesting, but unfortunately I haven't had the greatest luck in finding new songs I would enjoy listening to. It absolutely finds similar sounding tracks, but it doesn't distinguish which part of the song made it enjoyable. There's no tempo consistency or genre consistency or even main instrument/vocal timbre consistency between recommendations. I think locking one or more of those dimensions would allow for much better recommendations. I'm not sure what aspect you're using to order the results, but having extra metadata to filter or group the results in some way would help a lot.

Take Raga's Dance by Vanessa-Mae, A R Rahman, ... Royal Philharmonic https://maroofy.com/songs/476841571 . I put in this track expecting other fusion songs to pop up, and arguably some do, but much more often it feels like a 20 second section was used to define the original song and it misses the underlying concept. Like it got, in my subjective description, the epic violin in orchestral music, but it completely ignores the fusion between the distict styles of traditional indian singing/instrumentals and western ochestral and also ignores the call response structure between the violin and carnatic players, which is the what I actually care about. Other songs have the vocals but no epic backing. It feels like it's matching multiple samples from the song instead of the whole song.

This feels very promising since it clearly is picking up the styling of the specific songs across different genres and languages. I look forward to seeing where this goes.

I also think it would be interesting if there was a way to specify two different songs to find either only the common things and/or to find what the fusion of those two tracks produces.

Hey thanks for the feedback! I definitely have a lot of improvement to do on the model, it currently performs better for some styles/genres of music than others.

But the model architecture I'm using is kinda outdated as well, gotta iterate on it more to improve it further!

I'm also thinking of letting users upvote/downvote results, which can also help improve quality on the ranking side.

Honestly it's loads better than current Spotify/YouTube Music suggestions. Mostly they just seem to suggest popular stuff that's heavily marketed...even though I seeded all my "thumbs up" with only eclectic stuff.

Yes, it's hard to find a song I really really like, but 1-in-10 seem to be something I'd add to my eclectic "thumbs up" playlist. And almost none of them are by any artist that I've heard of before.

This is huge for me. Thanks.

You're not alone. For me, Spotify suggestions are "things you won't hate." Most everything is palatable, but forgettable and too usually not all that interesting.
I never get any heavily marketed music recommended on Spotify. Almost invariably it's something obscure. But I only ever listen to obscure music. I guess I'm saying I don't think the Algo is weighted for payola.
I honestly think they try first to make you happy, second to reduce their spend.
Wanted to hop on and say this is amazing, thank you for sharing this! Also agree that it seems that it's really good at finding literally similar sounding songs, but not what I would expect a friend to recommend (this is both good and bad I guess). As someone else said, this is already way better than my spotify recs
ty for ur kind words! <3
FWiW I had one shot and entered "Tabaran"

Rather than get back anything "acoustically similar" it simply returned a list of other songs on the same album (several of which are far from being acoustically similar).

No drama, you're attempting to cover a lot of ground, but I'm guessing there was no actual fingerprint there for that work and no sense of other songs that sounded similar.

ADDENDUM: Okay, I had to select the song <doh> .. but still "something went wrong" - perhaps hugged to death or not found to process. No matter :-)

I don't know if someone already said this, but as an amateur music producer i would love to upload my songs and discover similarities. Thanks for this Amazing tool
I'll note my own experience, that Spotify and Apple Music both struggle to find me latin reggaeton outside a small subset of popular artists, and my first couple searches with this tool have found me so much music I've never heard before that matches exactly the 'vibe' I want to hear, and is introducing me to different-but-related sounds and artists I couldn't have found on my own.

I agree with the other commenter - this is huge for me. Please, do whatever you need to do to monetize this so it never goes away. I would love to pay you for this.

I had the same experience. Could see the element which it matched with, beat, pitch, etc.but missed the riff or nuance that made the source song special to me
I am enjoying Raga's Dance, which is nothing like what I was just listening to. Thank you for the recommendation ;)
If I am not mistaken, it this is only trained on the preview and not on the entire song.

If you listen to a music with a real intro, it gives strange results. For example: "Goodbye Blue Sky - Pink Floyd" (https://maroofy.com/songs/1065976153)

Same for "Station by Station - David Bowie" -- lot's of tracks with ambient noise.
I agree with everyone's criticisms that it seems to identify similar tempo and melodic riff, irrespective of genre. But to me this is a feature, not a bug. I could see this or something like it opening my eyes to music I would never possibly have found on my own. I really like it!

Spotify on the other hand seems to want to send me to the same group of artists and tracks I've listened to before, following some Collatz conjecture type algorithm that eventually converges on the same tuned playlist for that genre, no matter what the starting parameters may be.

> But to me this is a feature, not a bug. I could see this or something like it opening my eyes to music I would never possibly have found on my own.

What makes it different from a big "play me a random song" button then?

They have some similarity based on the actual music.

> it seems to identify similar tempo and melodic riff

It’s a pretty cool idea and gets to a philosophical question really quick “what do people mean when they say they like similar music?”

Era? Artist? Genre? Sound? Tempo?

Personally I spend my time finding similar-era music because I like to hear how sounds evolved.

Ideally one would like an algorithm to be able to realize,"this person prefers to explore new music from the same era," vs "that person prefers to jump around to different countries," vs "the other person prefers to remix their existing playlists," and thus come up with the optimal degree of novelty for each listener. Or at least let the user set a novelty slider to customize their own experience.
Spotify wants you to listen to the tracks that they are paid to promote.
I've only tried a few songs but they've mostly been bangers! I did come across a couple examples where the recommended songs just heavily sampled the original but overall very impressed.
Categorising music is surprisingly different.

See this paper from https://everynoise.com/ : https://everynoise.com/EverynoiseIntro.pdf

IIRC they try to classify music on 17 different points/features. What you see on the web is an attenpt to visualise (and provide a guide to music based on) some of them

Yes. I think many of those features are based on pre-NN feature detectors (such as BPM), and Danceability, Valence and Energy sound like primary components that have been given names.

Echo nest was great for its time, but if they have kept up, they're not exposing their more modern learned features to users anymore.

They were acquired by Spotify, and there's been some work done by/for Spotify since then.

I'm not at liberty to say what, sadly, as I work for Spotify.

I think I can say that one of the main challenges is running this analysis for users. It's prohibitively expensive (or was prohibitively expensive) to use this to keep track of and run recommendations for what users are listening for each user.

It can be used on smaller scales, but, well, it's probably NDA :)

A distributed, local-first architecture much work well for this. I’m happy for my computer to crunch away on my behalf, generating recommendations and indexing stuff. I’m happy to recontribute that work to a common index of some kind.

I def prefer for that common index to have a permissive license though!

Can you say why Spotify's recommendations are so bad? Something like what OP has made should have been relatively simple to make for Spotify for many, many years already, yet that hasn't happen. Is the whole system just rigged to only recommended a few "sponspored" artists?
I doubt that "why does your product suck" is one of the things a Spotify employee is allowed to talk freely about in public!

But I've been watching them, I will speculate. A few years ago, Spotify had two young interns, Sander Dieleman and Aäron van den Oord. We know a bit of what they worked on, because Dieleman blogged on it, and indeed it was something a lot like what OP has made here - only better, I would say. I asked him, and Dieleman was allowed to say that the thing they built was one of the inputs into the then-new Discover Weekly, which made headlines for how outrageously good it was.

But Dieleman and v.d.Oord did not stay at Spotify. They were headhunted by DeepMind, and have had a VERY impressive track record there over the years.

And I wonder why. Was there a conflict between the old school ML of the Echo Nest people and the new fancy neural net kids? Or was it just, as GP alludes to, that the NN methods were just too computationally expensive and they failed to justify their costs to leadership?

Because, as I said above, it's a very complex problem :)

I honestly don't know much about recommendations (and what I know I probably cannot tell). But there's definitely continuous work done on them. But it can also be hampered by extremely conflicting requirements (where "some" both means double-digit procent of users and these "some"s overlap with each other):

- some users want more of the same, some users want a more diverse listening experience. Some of these users are the same user, but on different days

- some users mostly prefer curated suggestions, some users want ranodm stuff. They can also be the same user :)

- some users a heavily weigted to only a few artists, some users listen to evereything and anything. And even this can be the same user :)

- there's probably stuff about licensing, availability, contracts etc. at play as well, because in streaming services it's always there, in very bizarre ways

Basically every single tweak to recommendations will break them. And yeah, Spotify employees will complain about this more than anyone else, all the time :)

It doesn't seem to find similar-sounding tracks at all for me.

Examples:

The Oblio Joes - "Captain of the Moon"

The Bondage Fairies - "Levenus Supremus"

... both chosen so "Just shove a bunch of recent pop-rock at the user" won't work.

Isn't Apple going to give you a hard time now? Years ago, Apple might have offered to hire you. Given what a jerk is Tim Cook, I imagine he'll have the lawyers send a cease and desist.
I tried "learning to fly" by Pink Floyd. The results did not sound similar at all to me. Maybe I have non conventional idea on what is similar.
Tried with a song I knew well- Everything In Its Right Place.

Feels a little bit like fortune telling, I guess it is, in the sense that I am listening closely to what makes the songs similar, not just listening, but actively trying to find the similarities, so even a couple notes in progression, or drum-beats and I'll say oh, yes, that matches.

Finds very different music, not necessarily what I'd listen to in many cases, but kudos for getting me clicking through a decent pile before going wait, that's a nope, you're grasping AI-type-being.

I like the idea, but putting in some songs by Aphex Twin I didn't find the matches to be similar at all.
Interesting idea, I get pretty average recommendations though.

The first track I searched for was "Hazmat Modine - Bahamut" (https://maroofy.com/songs/253108933), it seems to recommend things with some similar instruments (e.g. brass and sax) but not really similar taste or style.