The setup is pretty basic. I've built a NextJS app, for the LLM model I am using open AI gpt-4-turbo and sending the images there directly without any database for images. I did a little prompting to get the same output everytime and when I get the output I make search on the Spotify API, find the songs and create the playlist with them on your own authenticated spotify account.
Likewise I also don't have a database for the emails eighter. I am using spotifys authentication
To keep your options open, it might be worth switching from GPT to either Gemini or Llama as OpenAI official policies prevent you from training on your logs with the argument this training is “illegal, harmful, or abusive,” so you’d never be able to fine tune or train your own AI in the future on your data or help others do the same.
If being permanently locked into a single intelligence service (or risking getting cut off or sued) is unacceptable for you as for me then OpenAI terms today are not acceptable. Yeah yeah maybe they won’t go after you, but why miss the opportunity for malicious compliance? Ditto Claude. Try a specialized model for your use case instead.
Gemini has no such customer noncompete, and with llama 3.1 meta removed theirs last week.
Thanks, that was something I was already considering. Do you know where can I find models specialized in this area? Because it is kinda niche, I couldn't find something that maps images into playlists directly, thats why i went for llms
This is great to hear, though I'm curious why photos on your phone / pinterest would be relevant to a recommendation system? Surely the biggest signal would be what Spotify already uses: the features of various relevant factors (your previous listening sessions, your current session, what other similar sessions look like, etc.), that said, their recommendation system is surprisingly terrible given how much easier music recommendations must be versus video, yet YouTube seems to have had this nailed for 15+ years whereas Spotify's "Discover Weekly" is so bad.
> I come up with an idea of generating playlists from images. Images that you shot on your phone yourself, or that you found on Pinterest, or a painting that you really like and feel inspired by.
This is genuinely interesting! Do you send the images to an LLM with a prompt like "generate a list of songs that would go well with this"?
Thank you! I should check youtube's algorithm too.
Yes, I have a prompt like that the current prompt is this:
'You match the vibes of the pictures with the right songs and turn them into a 3 song playlist with a playlist name. The music genre of the playlist should be consistent for each song. Be creative with music selections, explore different music, be consistent in terms of the genre of the 3 songs. The playlists should be provided in an object array format, like this: \'[{playlistName: "string", songs: [{songName: "string", artist: "string"}, {songName: "string", artist: "string"}, {songName: "string", artist: "string"}]}]\'. Do not add any other text information and only give outputs in the provided format. Your playlists must match the visual vibes and maintain the specified format without any additional information.',
Pretty basic, as i said before this was only ment for me and to explore this idea, but i loved it so I wanted to share :)
YouTube’s algorithm isn’t very good for users because it doesn’t really separate mildly interesting videos that you finish from awesome content you loved.
YouTube of course doesn’t care because they don’t make more money when you see something awesome.
That makes things worse IMO. My favorite videos tend to be one-offs not channels producing regular content. Unsubscribing from everything definitely improved my feed.
YouTube didn't help things any when they neutered the dislike button. It's still there but functionally useless. Yes, we all know why they did it to save the feelings of a few political staffers at campaigns aligned with the values of workers at Google but it's been an absolute godsend for scammers and terrible for signaling interests to the recommendation engine.
I think spotify's playlists are quite decent. I think they used to be a lot better than they are now, but I suspect that most recommendation systems decay over time. I suspect they don't handle the recommendation feedback very well, so they can start off introducing people to new things, but then become a bit more static and just reinforce the same habits over and over.
I think YouTube's recommendations used to be excellent, especially for music, but I've personally found it to be terrible recently. It no longer recommends anything new to me, and I suspect that it's way over-tuned. If I see a video that looks mildly interesting I'm a bit hesitant to watch it, because I don't want YT to decide that it should become 50% of my feed for the next week. Which from the recommendation system's perspective is just weakening the signal I'm feeding to it even further.
This is an interesting point and I wonder if part of the issue is that a mature, extremely popular algorithm trends towards the lowest common denominator. I don't mean this as a judgement of taste, it just seems to me that people engage with art in different ways. Maybe the spotify algorithm is perfectly tuned to the majority of people who just want to be able to find more songs that fit the kind of sound they like or find something to throw on in the background. But for a significant minority of others like myself and OP, it's just not tuned to what we actually want from new music.
I also feel the same as you regarding the youtube algorithm. I actually get better recommendations sometimes by just logging out since it will try showing me new stuff.
One thing I'm not sure about is whether it's actually the algorithm's fault or if my expectations have become unrealistic and made me lazy. I used to read magazines and blogs to find new music. There are still tons of people writing about their favorite music, labels that act as curators, etc. I just don't seek them out and instead expect to be spoonfed by the algorithms. Even if this is true though, I suspect many of these algorithms could do a better job.
Also RIP Netflix's old recommendation system. I guess it wouldn't make sense when they can't license every movie like they used to, but I remember it being great. Although maybe it was just pretty good and I was younger and less familiar with the back catalog of good films.
That's the other thing I wonder - am I just getting older and less excited about new things? There used to be a real vitality to finding something new and exciting. Now it kind of feels hard for anything to feel that fresh anymore, it all seems like variations on the same core ideas. I do still find new stuff that I like, but it doesn't have the same thrill. Maybe I'll always be chasing that dragon of youth haha.
Unfortunately it's an extremely unprofitable industry, with very little revenue in general, and that little revenue shared by a few duopolists. Although it's an art, it's capitalistically treated like a craft, successful practitioners serving the same, safe, "good-enough", risk-free, pleasant sound again and again. I like offensive, avant-garde, creative, novel, strange music and (1) artists I love live in immense poverty (i.e. artist life) (2) even though this is the biggest passion of my in life and I think I do have some skills, working on music is 99.999...% surefire way to have financial hardship.
I don't think he wants a passport photo, just saying any picture as a seed — just enter a photo of a dog (so long as it's not your "first pet" with a personalised name tag round her neck)
The setup is pretty basic. I've built a NextJS app, for the LLM model I am using open AI gpt-4-turbo and sending the images there directly without any database for images. I did a little prompting to get the same output everytime and when I get the output I make search on the Spotify API, find the songs and create the playlist with them on your own authenticated spotify account.
Likewise I also don't have a database for the emails eighter. I am using spotifys authentication
Music discovery has never worked for me, for the simple reason it's the lyrics, not the music. I listen to anyone speaking truth, (the truth I believe, of course), and that gives me a wide disjoint range of music, but they are all singing about political social truths. Marvin Gaye, Public Enemy, Rage Against The Machine, Beyonce, The Stranglers, The Jam, Psychic TV... It's the lyrics, and now today, we finally have the ability to have music discovery with the lyrics, with the intellectual content of the music and not just the dressing.
That's a very good point, I also love discovering with lyrics. What I tried to do is to find some connections between the image and music, maybe some connection we as humans cannot see right away.
I only pay attention to melodies and rarely can even discern the lyrics unless I have them written out as I listen. Even relatively clean vocals like Johnny Cash just wash over me without being understood. The words for me become just another instrument that can play notes. If I can always predict the next series it is boring, if I never can, it is too challenging. In the middle I get dopamine whether I predict the notes or not. The right series of notes can make me feel such varied emotions with no words necessary.
Music discovery used to work for me with pandora, nothing else has. I have no idea if they had better algorithms, or just a better catalog. It doesn't seem to work as well as it once did either.
Tl;dr I don't think it is simply your preference for lyrical content over melodic content that causes the algorithms to fail. They are just bad.
You might like folk punk - check out Pat the Bunny.
I feel like there are two kinds of singers - people who are good at singing, and people who have something to sing about. You and I, I think, prefer the latter.
Even though you'r being mean I feel the need to defend you, only because I agree that this was one of the most unrelatable things I've ever read on the site (i.e. GP's preference for music with a political or social theme in the lyrics rather than a specific genre, theme, sound, style, or mood).
If you're reacting to the inclusion of Beyonce, I suggest checking out her recent Country music album, which is a political re-envisioning of Country music and to many a direct attack on the comfort seat of White racism.
Many of you asked how it looks, how it works. I've added a video under the blog post, showing how it works.
Many of you were also interested in knowing what's going on behind the scenes. The setup is pretty basic. I've built a NextJS app, for the LLM model I am using open AI gpt-4-turbo and sending the images there directly without any database for images. I did a little prompting to get the same output everytime and when I get the output I make search on the Spotify API, find the songs and create the playlist with them on your own authenticated spotify account.
Likewise I also don't have a database for the emails eighter. I am using spotifys authentication.
As I said before this was just for me at first but it is very exticing to see many people interested by this idea.
If you don't trust openAI models and don't wanna send any pic data to them please don't write your email
In the age of internet, engagement optimization and recommendation algorithms create a new way that we are affected by the behaviour of others.
That annoying dark pattern on a piece of software you use? Because there are people who fall to it, clicking on an ad or "engaging" more. That stupid show that keeps being recommended to you? Because a lot of people just sit on the couch, watching something on the list that does not need too much mental processing.
I have a peculiar taste in Music. I love many many different types of music, but once I find a really good piece, I'm not interested in things that are very similar to this one. Looks like if we describe musical work with high dimensional vectors, I like to find good vectors that are not too close to each other. But as the author said, Spotify keeps showing me music that's similar to what I've listened. That's exactly what I don't like (with the occasional exception of something being better than the one I've already found and replacing it).
I assume I belong to a peculiar minority. The recommendation algorithms work very well for predictable majorities.
Maybe someday we have an interesting "musical embeddings" model, and then people can implement personalized discovery algorithms using that?
Yess! Thank you for commenting. I am very interested in this topic. Please do share with me if you find any interesting ways to explore new music for your taste
Same as GP. It takes times. Unless it’s party mode, I only listen to albums. To find new music, it takes time mostly. I decide to listen to a new genre and I seek a playlist or a compilation curated by someone. If I find someone I like, I check their albums. I also checkout the recommendations on bandcamp (people vote with their wallet there). Then there are forums and polls, and I may decide to try something out of the blue.
The more you curate, the more you define your own taste. It’s then easier to describe what you like in a music and triage.
local record store. talk to the people who work there, tell them the 5 albums you’ve been hooked on lately.
at the end of the day no matter how many times we beat our heads into the same wall, we’re not even close to an accurate discovery model, music nerds are far better at recommendations than any discovery models. far better.
don’t let their insufferability discourage you. you will be too once you start diving into and going on rants about music which is outside of mainstream fluff. it’s like this with any $subject involving wonks. we’re insufferable to anyone who isn’t into our particular genre of technology. food wonks are insufferable, car geeks are insufferable, gamers are insufferable. that’s ok, if you’re looking for someone who is a geek in a topic, you’re likely to become one too :p just be normal around $subject non-wonks and you’ll be fine.
but yeah, music nerds working in a good record store really do know their stuff.
other places:
- music nerd streams on twitch
- music reviewer youtube channels
- college radio stations (most have an online presence) 770 radiok out of minneapolis is incredible
- kexp out of seattle is absolutely amazing (they’re heavily online as well.)
- just about every mid+ sized city has some amazing radio, usually found in the low FM areas.
at the end of the day though, it’s other people. there are far too many variables for every individual which drives why they may or may not like a song at any given moment. other humans are still absolutely unmatched when it comes to navigating this.
> local record store. talk to the people who work there, tell them the 5 albums you’ve been hooked on lately.
Unless your interests are niche. A fun game I used to play as a teen was going with my parents to the record store and seeing if they had any music I listened to online while my parents shopped. Never found a single CD (but they couldn't be that niche, this story is about bands I found out about from my friends at school!). Employees tried to be helpful, but there's only so much they can do when someone comes in and asks for a list of bands they've never heard of.
This probably won't work for most people, but as I mostly like metal, I simply check all metal releases that seem slightly to my tastes according to the subgenre. Then when this sampling seems at least somewhat positive, I play the full album after release. That usually ends in sampling songs from 30 albums, listening to ten albums, and buying 1-2 every Friday. Never had an algorithmic recommendation system that worked for me.
If you have any taste at all other than "maximally dissimilar to anything I have liked before," there should be a feature that predicts songs you would like.
If your taste is exactly "maximally dissimilar to anything I have liked before," that's actually pretty easy to calculate from the embeddings as well.
It didn't sound like op wanted maximally dissimilar from what they've liked before, but instead maximally dissimilar from what they've listened to recently.
I'm not sure how the Spotify recommendation algorithm works at all, but for some reason I imagined them doing fancier things than looking at my liked songs and finding similar ones. I would've thought they'd build a profile of you, and then find similar user profiles and show you songs those folks liked that you hadn't found yet.
That's gotta be how they do it, right? I'm probably wrong.
I feel those are how Pandora and Last.fm (used to?) work respectively. Nowadays everything seems to just put a bunch of tags on a track and suggest you things with the same tags to the tracks you liked. Doesn't even need to match the same combination of tags, just some number of them. The problem is, you probably care about the small, specific tags, and the system cares about wide "popular" tags. If you like a couple niche genre covers of songs that happen to be featured in TV openings/OSTs, you are not getting more songs in that genre - you are getting a bunch of covers and OSTs.
I wish I had a music recommendation service built on Pandora's immense dataset of music tags that could build me a playlist that I could link back to whichever music service I happen to be using at the time. I could have it do things like require at least 3 tags in common between adjacent tracks such that it could jump around between 2 dozen genres but the transaction between any 2 given tracks isn't too jarring. It'd also be nice if I could tell it to make a playlist where every song shares one particular tag in common.
Maybe I'll build that. Sure would be nice to have.
The primary advantage of Pandora's algorithm is the human-labelled Music Genome database. I haven't seen any other company do music discovery as well as Pandora, and don't expect that to change any time soon.
I have no specific insider knowledge, but a decade+ ago they bought a company called the Echo Nest that was developing some of the best audio signal analysis algorithms around, I assume much of that influenced their recommender system.
Nowadays, they have a quite busy research department so I would imagine that recommendation is quite fancy indeed: https://research.atspotify.com
I don't work at Spotify anymore and I didn't work on the tech I'm describing, but I picked up a bit about what was going on while there.
First, there is/was no single algorithm, but the core ideas driving a lot of recommendations is:
1. Create user taste vectors
2. Match those vectors to other users or collections of tracks
3. Use that information and combinations of other things to find recommendations.
Each step of the process is constantly being experimented with. Different custom playlists might be using a different combination of tech doing those basic steps.
Collaborative filtering is similar but for huge recommender systems they’re not going to create a huge MxN matrix where M is users and N is items. I think what they’re referring to would be called a “two tower” model where you have a learned vector for the user, a learned vector for the song, and the cosine similarity is their affinity. It’s pretty performant because you can cache the song vectors.
"Other things" including intentional commercial biases presumably?
No matter what I do in Spotify, under several different rounds of accounts, it always seems to gravitate towards the tastes of the general public, i.e. some form of mass-market pop.
Their recent "ai" assistant was a slight improvement because you can ask it for less popular music which is typically better for music discovery.
Instead of relying on things like "Playlist made for you" I still like leveraging the algorithm to discover "new vectors" by actively going to a song that some aspect caught my attention and going to its radio (only tried on Spotify). Sometimes I'm surprised with new soundscapes.
This is one of my main complaints. Back when Google Play Music was still a thing, I uploaded my library of ~30k tracks that had been ripped from the massive binder of CDs I'd accumulated over the years, plus stuff from friends, stuff I got off torrents to replace albums I'd already bought on tape, etc.
They have a huge sample set of stuff I like just from that (it's still accessibly via YouTube Music, but it sucks). Still, any time I fire up YouTube Music and play a song, the next 2 or 3 can be solidly appropriate and often stuff I wouldn't have thought to play - in a good way.
Then there's a sharp decline where it just starts playing the same thing it played last time or the most common song off the most popular album from the most well known artist adjacent to what played previously.
The whole reason I use the "algorithmic feed" is because it should be optimized for my tastes and what will keep me listening. I still "thumbs-up/down" stuff but it never seems to work. You would think with their insane library and huge dataset of my prefs and listening data they'd be able to generate something great.
Instead I just go back to the human programmed Shoutcast stations I have bookmarked after a few songs.
The same for me. Nowadays i make my own songs using Udio and i upload 'em to YT. It is unlikely i will ever listen to suggested songs ever again, by any service.
> I'm not interested in things that are very similar to this one.
Are you sure this is an accurate description of your taste? Or do you mean "I'm not interested in things that are similar, but lower quality"?
I'm very much a weird-music enjoyer, and I often have the latter problem where "similar" songs actually just don't capture the same vibe as the truly engaging new song I just heard. But that's not because the algorithm is choosing music that's too similar; it's the opposite. It's trying to choose something similar but can't, so it just picks the next-best thing which I actually don't like.
I agree strongly. Listen to the radio stream for any reasonably niche artist (especially if they have had a single "hit") and you will be treated to 3 or 4 different mixes of the same song within the first hour. For some purposes Spotify is just unlistenable.
No music discovery algorithm has satisfied me. All data-driven approaches make predictions based on historical data. Personally I enjoy being exposed to entirely new genres and sounds I've never heard before, instead of variations on genres I've listened to a lot.
My solution: listening to NTS, an eclectic online radio station, where diverse artists create playlists.
If entirely new things is what you're looking for, you're not really looking for a recommendation algorithm [1]. What these algorithms try to achieve is finding unknown songs that are in the same genre to what people already like.
[1] Technically "random song not in listen history" would work out, if you'd really like to call that a recommendation algorithm.
But I also don't want a totally random song eighter. I want something that vibes with me but not directly recommended through my listening history, because then they are extremely similar and feels like they're feeding me the same melodies over and over.
Thats why I tried to give the "vibes" in a different format; image, rather than my listening history.
I agree that NTS radio is one of the best ways to be exposed to interesting new and old music, obscure stuff, brilliant mixes, etc.
The NTS app is great: for Web, Android, iOS - it's always being steadily improved. A very nice feature to aid discovery/curation is that every track in a tracklist has a 'copy song and artist info' so you can easily search for tracks on your streaming platform. Not sure if this is a subscriber only feature.
I also use the 'identify song' feature in the Google search app on my phone, similar to Shazam.
If the algorithms aren't doing it for you then do yourself a favor and head to https://nts.live
Interesting. For example how would one find a "new" Tool or Deftones? Current algorithms probably don't "pick up" not-yet-so-popular things. For example Shelton San (I found out about them via word-of-mouth), although I'm frequent user of Spotify.
This means that classical promotion channels are still necessary, as otherwise things get lost in noise.
I noticed that Spotify surfaces similar artists who are also of a similar popularity. So it's not like it doesn't understand that particular style, it just has to somehow pick a couple of dozen artists to show in that very coveted spot.
So what worked for me in the past is finding less popular artists and then checking their similar artists.
The most interesting thing in this, to me, is just how differently people perceive and enjoy music. I find that the algorithms work pretty well for me. But novelty in melody or rhythm are not something I care about (I literally can not remember an instance of even thinking about how a melody in a new song might go, let alone predicting it). The kind of “newness” that the algorithms provide work exactly right for how I enjoy music. But it makes a ton of sense that if you are more driven by finding new melodies, they would not work well for you.
Even though the idea of recommendations is anything but new, literally nothing and nowhere works as expected. The only thing that comes close is based on the concept of neighbours, as implemented at Last.fm or RateYourMusic.
I don't understand why is it so hard to offer something along these lines:
1. Define dominant user preferences by clustering and segmenting the field of listened genres.
2. Build a list of relevant "neighbours":
2.1 Manually added users/friends
2.2 For each of the dominant genre preferences, find users with a high level of artist intersection within that genre and add them
3. Now, for a "find similar" query:
3.1 Define a reasonable time window
3.2 For each neighbour, find points in time when they listened to the queried track/artist
3.3 Build a list of tracks/artist from the defined window around the points found
3.4 Filter tracks/artists that are too "distant" on the general genre/tag map, or lie outside of the user's dominant preferences (with a degree of boundary feathering, perhaps)
3.5 Filter if similar to negative part of the query
3.6 If novelty is required: filter artists/tracks according to the degree of their presence in the user's history
I think the financial incentives to promote specific artists (or songs with ads in them "GUCCI!") become the focus pretty quickly.
The cost to play a song is expensive, so if you can actually profit by putting a new artist instead of paying, why wouldn't you?
Sure your customer gets 3 minutes of potential garbage, but they don't realize that they generated revenue for the company just by sitting through that song.
If you give your customers a great experience, they are going to listen to more music, which is bad for the bottom line.
There doesnt seem to be any competition due to IP laws, so there is no incentive to be good.
1. The curse of dimensionality when computong distance functions and
2. The cost function of a bad song. People get mad if they get too many things they don't like. Notice Pandora immediately sends you back to the band that seeded a station upon any thunbs down.
"They are not suggesting new, very interesting melodies. They are finding you the tweaked versions of the songs you already like and, even on your first listen you can predict the melody that’s to come."
I really don't think that's the main method of Apple Music or Spotify to create a list of suggestions. From what I know, (beside of dark marketing-patterns) the suggestions are created by checking what other songs people like/listen to who ALSO like/listen to this current song (or other songs you played), and the common neighbors of those songs in other playlists.
(If you play music for your toddler, your future suggestions will include children's music not because it sounds similar but because "a critical mass of other people who listened to Baby Shark on repeat also listened to: Old Town Road")
It is weird and it’s ironic that they call that “discovery”, as it feels more like variations of what I'm already listening to.
This indicates that the persona that this platform created for you is quite homogenous and probably matches closely with many other personas on the platform, so many people who listen to the same music as you do apparently listen to _nothing else_ than this kind of music...
(not trying to defend those suggestion algorithms, just analyzing the comment)
> This indicates that the persona that this platform created for you is quite homogenous and probably matches closely with many other personas on the platform
To add to this analysis, I think there may also be a feedback component to this problem that exacerbates the issue, since most users are passively using the suggestion algorithm.
In other words, if the suggestion algorithm tends to create a homogenized persona of the user's taste, say, because they don't bother to actively correct it, then this persona is embedded into a cluster of people with similar personas. And because the persona is now closer to said cluster, the suggestions will become even more homogenized. Moreover, since the cluster is mostly composed of passive users, the cluster itself will tend shrink (eg in variance) and to get more homogeneous.
I suspect that most algorithms do not do enough to prevent this global trapping effect, and so even if they have some method to sample "something new" for the user this becomes less and less efficient as more users rely on the algorithm for their suggestions.
I actually do find this observation to be quite accurate for many of my own 'suggestions.' I'm regularly recommended 'new' and 'old' music that was clearly matched to my 'tastes' only by melody or, more noticeably, sample. It very much seems like if a song fits into a genre I listen to frequently or have been listening to lately, and it samples another song I've listened to before—cheap recommendation. And the greater the frequency of individual plays (i.e. the more times I've replayed any one song), the more likely that derivatives will be recommended to me.
It's easy to see how this would've been baked into a human-made algorithm when you consider waveforms. Speaking only to Spotify's algorithm here. And it doesn't really bother me for obvious reasons. But it is creating something of a musical echo chamber for me.
Radio Paradise is very much a rock station at heart, so necessarily for everyone's liking. If you're into classic rock mixed with contemporary rock, mixed with a bit of everything else, it's worth a shot.
In my experience literally anything beats that one friend who is a DJ. My friends - professional DJs in Berlin - haven't even heard of Can, to my absolute shock.
The “and still obsessively digs crates” was important I think. DJ’ are not even close to fungible (hell, I’ve even met a couple who don’t like music much)
All music recommendation engines at this time still aspire to be mediocre, they aren’t even playing the S.A.,e game as a human who is good at it.
Unfortunately, such humans are unevenly distributed.
My point is more about DJ specialization I guess. The vast majority of DJs I know personally still dig crates all the time - but they are techno/house/D&B/etc DJs, they know close to nothing about music outside of these genres. This goes so deep that some German techno DJs haven't even heard of Krautrock, the German scene that in many ways was a precursor of electronic music.
German beat-oriented DJs haven't heard of one of the most influential German bands that had a critical impact on beat-oriented music? Just an odd thing for people who dedicated their life to music.
I really wish Spotify or Apple offered the ability for the listener to simply listen all of the songs released on their platform on a day or a week, good or bad, and directly pay the artists for the songs listeners enjoy.
Spotify's "New Releases" for example, tracks only music that major labels promote, or that fit a predetermined genre, or are similar to the songs and artists that you already listen to.
There are smaller services yes, which allow for independent promotion and distribution (Last.fm, RateYourMusic) but these have fairly obvious flaws in how the listener can approach new music (RYM pushes ratings first and foremost, and both last.fm and rym push trending artists to their users).
Instead, because the value of music is zero (really, negative since the number of listens, streams, album purchases, etc can fail to recuperate the cost to make it ), the act of distributing music presents economic risk unless the release itself can be controlled by the investors through advertisement or paid promotion.
The real big lie is the idea that the recommendation algos were ever actually for the user.
The premise that many folks miss here is the idea that Spotify is, at best, thinly interested in recommending music that is good for YOUR interests. Spotify is the music business, and specifically the pop music business, has long discovered that's it's much more economically expedient to force feed musical taste onto the public than it is to chase the whims of organic hit-making. Payola is as old as recorded music. Spotify recommends what Spotify wants it's users to listen to. They have all kinds of side deals and marketing deals with labels, they have cheaper costs/royalties on some tracks than others. Popular tracks cached in their CDNs are probably cheaper to recommend than long tail ones etc. They have strategic priorities like gaining on apple for podcasts, and therefore injecting allsorts of podcast recos in the UI wether you asked for that or not.
I figure the problem is access. You either have the obscure music people want to discover in your db or you don't. The music available on spotify is a drop in the ocean so it doesn't really matter how searches work because a majority of the possible results simply aren't known by the platform
Think they also direct listeners to cheaper to license sound-alike version of songs, especially from previous decades. I've pretty much given up on the recommendations from any of these companies. Pandora used to be reasonably good but they started playing the Studio 54 game where there was always another higher level of subscription to buy to avoid annoyances they would create. The best recommendation engine I've used was last.fm for the xbox. I could leave that on all day and rarely need to do a skip. But that was discontinued long ago, maybe ten or fifteen years ago? Haven't seen anything come close since. Amazon keeps giving me free Amazon Music but it's not even worth the bother of loading up as the system is so focused on anything and everything except my musical tastes.
Personally all I want out of a subscription based music service is excellent quality, constantly updated/created, thematically consistent, human curated playlists. I don’t really care if it’s super popular or fringe stuff, but I do want it to be as “good” as the other stuff on the playlist, and I want a human who also cares about the music to be making that decision. Sometimes the playlists in Apple Music scratch this itch, but it would be amazing if they were constantly updated.
I've got a Tidal subscription, and I've played with creating a few of my own playlists for the explicit purpose of sharing with others (as opposed to personal use). They're nothing special, but I at least try to put in some research while constructing them to bound them at a narrow time-frame/genre for accurate historical purposes.
I have no idea if anyone is listening to them, though, because there doesn't seem to be any feedback system for the community playlists. That would be a useful addition, IMO. If TIDAL doesn't want to pay dedicated staff to curate playlists, they could at least make some way for the member-created playlists to get featured or gain reputation.
Deezer has human curated playlists updated frequently, which has been my main way of discovering new artists these days. For example, the prog metal playlist as been updated 5 days ago. [0] There are lots of those playlists..
I don’t know if it’s the case anymore but I used to use last.fm’s radio service 8 or so years ago and it was absolutely great at finding and recommending me genuinely new stuff.
> They are not suggesting new, very interesting melodies. They are finding you the tweaked versions of the songs you already like and, even on your first listen you can predict the melody that’s to come
This seems like the complaint of somebody who hasn't been using spotify very long. After a decade plus, I feel like my algorithm is a rich compost pile of all of my previous phases of music. Spotify is excellent at letting me broaden my horizons or jump down a rabbit hole from a random starting point, like a song I hear in a public space or commercial or something sent by a friend. Maybe the OP should keep their ears open to more sources of randomness from the outside world?
I feel the opposite: my Spotify recs (after at least 8 years with an account) tend to get stuck on whatever I've been listening to recently. I've had to consistently go afield to find any new (to me) music. Even their "new releases for you" falls short of recommending me releases from artists I follow. How much less capable could it be?
Release Radar is consistently the worst feature of Spotify. It misses entire new albums from artists I listen to regularly, and seems to have a quota of songs to fill so after the first two or three it's no longer aligned with my interests. I can forgive it not being coherent since it's supposed to include multiple genres together, but I can't forgive it going way off from what I like just to hit 30 songs.
Huh, I don't even have that section on my Spotify. I have a "New music you need to hear this week" at the very bottom (none of it is anything I need to hear this week), but it's just generic "new music in X genre" playlists.
This is my experience as well... I have a very broad music taste but with some main themes. I find Spotify's algorithm (11 years of Premium) to regularly surface things I'll like, whether new music from artists I already know, music correlating strongly with known tastes, or every once in a while something that seems out of distribution but I like it anyway!
It probably helps that the strongest areas of my taste are relatively small or niche genres, like Scottish trad and Celtic (folk) rock. In those niches, similar-but-different is often distinctively different in actual experience. Sure, there's covers of the same song from time to time, but I actually do like enough of those not to be bothered, if they bring something new.
197 comments
[ 3.2 ms ] story [ 243 ms ] threadLikewise I also don't have a database for the emails eighter. I am using spotifys authentication
If being permanently locked into a single intelligence service (or risking getting cut off or sued) is unacceptable for you as for me then OpenAI terms today are not acceptable. Yeah yeah maybe they won’t go after you, but why miss the opportunity for malicious compliance? Ditto Claude. Try a specialized model for your use case instead.
Gemini has no such customer noncompete, and with llama 3.1 meta removed theirs last week.
> I come up with an idea of generating playlists from images. Images that you shot on your phone yourself, or that you found on Pinterest, or a painting that you really like and feel inspired by.
This is genuinely interesting! Do you send the images to an LLM with a prompt like "generate a list of songs that would go well with this"?
Yes, I have a prompt like that the current prompt is this:
Pretty basic, as i said before this was only ment for me and to explore this idea, but i loved it so I wanted to share :)YouTube of course doesn’t care because they don’t make more money when you see something awesome.
That makes things worse IMO. My favorite videos tend to be one-offs not channels producing regular content. Unsubscribing from everything definitely improved my feed.
I think YouTube's recommendations used to be excellent, especially for music, but I've personally found it to be terrible recently. It no longer recommends anything new to me, and I suspect that it's way over-tuned. If I see a video that looks mildly interesting I'm a bit hesitant to watch it, because I don't want YT to decide that it should become 50% of my feed for the next week. Which from the recommendation system's perspective is just weakening the signal I'm feeding to it even further.
I also feel the same as you regarding the youtube algorithm. I actually get better recommendations sometimes by just logging out since it will try showing me new stuff.
One thing I'm not sure about is whether it's actually the algorithm's fault or if my expectations have become unrealistic and made me lazy. I used to read magazines and blogs to find new music. There are still tons of people writing about their favorite music, labels that act as curators, etc. I just don't seek them out and instead expect to be spoonfed by the algorithms. Even if this is true though, I suspect many of these algorithms could do a better job.
Also RIP Netflix's old recommendation system. I guess it wouldn't make sense when they can't license every movie like they used to, but I remember it being great. Although maybe it was just pretty good and I was younger and less familiar with the back catalog of good films.
That's the other thing I wonder - am I just getting older and less excited about new things? There used to be a real vitality to finding something new and exciting. Now it kind of feels hard for anything to feel that fresh anymore, it all seems like variations on the same core ideas. I do still find new stuff that I like, but it doesn't have the same thrill. Maybe I'll always be chasing that dragon of youth haha.
Plus, there is no real explanation on the page as to how this would work, not even from a high level.
Likewise I also don't have a database for the emails eighter. I am using spotifys authentication
Music discovery used to work for me with pandora, nothing else has. I have no idea if they had better algorithms, or just a better catalog. It doesn't seem to work as well as it once did either.
Tl;dr I don't think it is simply your preference for lyrical content over melodic content that causes the algorithms to fail. They are just bad.
I feel like there are two kinds of singers - people who are good at singing, and people who have something to sing about. You and I, I think, prefer the latter.
> Marvin Gaye, Public Enemy, Rage Against The Machine, Beyonce
Thanks for the laugh, man, I needed it.
Many of you were also interested in knowing what's going on behind the scenes. The setup is pretty basic. I've built a NextJS app, for the LLM model I am using open AI gpt-4-turbo and sending the images there directly without any database for images. I did a little prompting to get the same output everytime and when I get the output I make search on the Spotify API, find the songs and create the playlist with them on your own authenticated spotify account.
Likewise I also don't have a database for the emails eighter. I am using spotifys authentication.
As I said before this was just for me at first but it is very exticing to see many people interested by this idea.
If you don't trust openAI models and don't wanna send any pic data to them please don't write your email
That annoying dark pattern on a piece of software you use? Because there are people who fall to it, clicking on an ad or "engaging" more. That stupid show that keeps being recommended to you? Because a lot of people just sit on the couch, watching something on the list that does not need too much mental processing.
I have a peculiar taste in Music. I love many many different types of music, but once I find a really good piece, I'm not interested in things that are very similar to this one. Looks like if we describe musical work with high dimensional vectors, I like to find good vectors that are not too close to each other. But as the author said, Spotify keeps showing me music that's similar to what I've listened. That's exactly what I don't like (with the occasional exception of something being better than the one I've already found and replacing it).
I assume I belong to a peculiar minority. The recommendation algorithms work very well for predictable majorities.
Maybe someday we have an interesting "musical embeddings" model, and then people can implement personalized discovery algorithms using that?
The more you curate, the more you define your own taste. It’s then easier to describe what you like in a music and triage.
at the end of the day no matter how many times we beat our heads into the same wall, we’re not even close to an accurate discovery model, music nerds are far better at recommendations than any discovery models. far better.
don’t let their insufferability discourage you. you will be too once you start diving into and going on rants about music which is outside of mainstream fluff. it’s like this with any $subject involving wonks. we’re insufferable to anyone who isn’t into our particular genre of technology. food wonks are insufferable, car geeks are insufferable, gamers are insufferable. that’s ok, if you’re looking for someone who is a geek in a topic, you’re likely to become one too :p just be normal around $subject non-wonks and you’ll be fine.
but yeah, music nerds working in a good record store really do know their stuff.
other places:
- music nerd streams on twitch
- music reviewer youtube channels
- college radio stations (most have an online presence) 770 radiok out of minneapolis is incredible
- kexp out of seattle is absolutely amazing (they’re heavily online as well.)
- just about every mid+ sized city has some amazing radio, usually found in the low FM areas.
at the end of the day though, it’s other people. there are far too many variables for every individual which drives why they may or may not like a song at any given moment. other humans are still absolutely unmatched when it comes to navigating this.
Unless your interests are niche. A fun game I used to play as a teen was going with my parents to the record store and seeing if they had any music I listened to online while my parents shopped. Never found a single CD (but they couldn't be that niche, this story is about bands I found out about from my friends at school!). Employees tried to be helpful, but there's only so much they can do when someone comes in and asks for a list of bands they've never heard of.
If your taste is exactly "maximally dissimilar to anything I have liked before," that's actually pretty easy to calculate from the embeddings as well.
That's gotta be how they do it, right? I'm probably wrong.
Maybe I'll build that. Sure would be nice to have.
Nowadays, they have a quite busy research department so I would imagine that recommendation is quite fancy indeed: https://research.atspotify.com
https://www.canburypress.com/products/you-have-not-yet-heard...
First, there is/was no single algorithm, but the core ideas driving a lot of recommendations is:
1. Create user taste vectors
2. Match those vectors to other users or collections of tracks
3. Use that information and combinations of other things to find recommendations.
Each step of the process is constantly being experimented with. Different custom playlists might be using a different combination of tech doing those basic steps.
[Disclosure: Work at Google, but not on that. Just thought that course was particularly well-designed.]
No matter what I do in Spotify, under several different rounds of accounts, it always seems to gravitate towards the tastes of the general public, i.e. some form of mass-market pop.
Their recent "ai" assistant was a slight improvement because you can ask it for less popular music which is typically better for music discovery.
Maybe these music services should ask you for music you hate, and start from there instead.
They have a huge sample set of stuff I like just from that (it's still accessibly via YouTube Music, but it sucks). Still, any time I fire up YouTube Music and play a song, the next 2 or 3 can be solidly appropriate and often stuff I wouldn't have thought to play - in a good way.
Then there's a sharp decline where it just starts playing the same thing it played last time or the most common song off the most popular album from the most well known artist adjacent to what played previously.
The whole reason I use the "algorithmic feed" is because it should be optimized for my tastes and what will keep me listening. I still "thumbs-up/down" stuff but it never seems to work. You would think with their insane library and huge dataset of my prefs and listening data they'd be able to generate something great.
Instead I just go back to the human programmed Shoutcast stations I have bookmarked after a few songs.
It's the classic problem of "Ok, but how do I sort these by which ones are actually any good?!"
Are you sure this is an accurate description of your taste? Or do you mean "I'm not interested in things that are similar, but lower quality"?
I'm very much a weird-music enjoyer, and I often have the latter problem where "similar" songs actually just don't capture the same vibe as the truly engaging new song I just heard. But that's not because the algorithm is choosing music that's too similar; it's the opposite. It's trying to choose something similar but can't, so it just picks the next-best thing which I actually don't like.
My solution: listening to NTS, an eclectic online radio station, where diverse artists create playlists.
[1] Technically "random song not in listen history" would work out, if you'd really like to call that a recommendation algorithm.
https://www.barneyhill.com/pages/nts-tracklists/
The NTS app is great: for Web, Android, iOS - it's always being steadily improved. A very nice feature to aid discovery/curation is that every track in a tracklist has a 'copy song and artist info' so you can easily search for tracks on your streaming platform. Not sure if this is a subscriber only feature.
I also use the 'identify song' feature in the Google search app on my phone, similar to Shazam.
If the algorithms aren't doing it for you then do yourself a favor and head to https://nts.live
So what worked for me in the past is finding less popular artists and then checking their similar artists.
Please get in touch
I don't understand why is it so hard to offer something along these lines:
The cost to play a song is expensive, so if you can actually profit by putting a new artist instead of paying, why wouldn't you?
Sure your customer gets 3 minutes of potential garbage, but they don't realize that they generated revenue for the company just by sitting through that song.
If you give your customers a great experience, they are going to listen to more music, which is bad for the bottom line.
There doesnt seem to be any competition due to IP laws, so there is no incentive to be good.
1. The curse of dimensionality when computong distance functions and
2. The cost function of a bad song. People get mad if they get too many things they don't like. Notice Pandora immediately sends you back to the band that seeded a station upon any thunbs down.
(If you play music for your toddler, your future suggestions will include children's music not because it sounds similar but because "a critical mass of other people who listened to Baby Shark on repeat also listened to: Old Town Road")
This indicates that the persona that this platform created for you is quite homogenous and probably matches closely with many other personas on the platform, so many people who listen to the same music as you do apparently listen to _nothing else_ than this kind of music...(not trying to defend those suggestion algorithms, just analyzing the comment)
To add to this analysis, I think there may also be a feedback component to this problem that exacerbates the issue, since most users are passively using the suggestion algorithm.
In other words, if the suggestion algorithm tends to create a homogenized persona of the user's taste, say, because they don't bother to actively correct it, then this persona is embedded into a cluster of people with similar personas. And because the persona is now closer to said cluster, the suggestions will become even more homogenized. Moreover, since the cluster is mostly composed of passive users, the cluster itself will tend shrink (eg in variance) and to get more homogeneous.
I suspect that most algorithms do not do enough to prevent this global trapping effect, and so even if they have some method to sample "something new" for the user this becomes less and less efficient as more users rely on the algorithm for their suggestions.
It's easy to see how this would've been baked into a human-made algorithm when you consider waveforms. Speaking only to Spotify's algorithm here. And it doesn't really bother me for obvious reasons. But it is creating something of a musical echo chamber for me.
Nothing beats that one friend who used to DJ and still obsessively digs crates.
Radio Paradise is very much a rock station at heart, so necessarily for everyone's liking. If you're into classic rock mixed with contemporary rock, mixed with a bit of everything else, it's worth a shot.
that dj friend or like i said in a different comment, your local record store employees.
college radio stations.
and just other people. it really is that simple.
People who feel they have a calling to be a DJ sometimes actually do.
I have a reasonable list of TuneIn stations (mostly US) that provide my favorite “discovery”.
All music recommendation engines at this time still aspire to be mediocre, they aren’t even playing the S.A.,e game as a human who is good at it.
Unfortunately, such humans are unevenly distributed.
If you like old-school metal, here's some great youtube channels (no affiliation to myself):
https://www.youtube.com/channel/UCCGbKiCJjph8Grazqmo7z4w https://www.youtube.com/channel/UCD5Ny_jQ8cs9JXVPWXg9iNw
Support these small bands.
There are smaller services yes, which allow for independent promotion and distribution (Last.fm, RateYourMusic) but these have fairly obvious flaws in how the listener can approach new music (RYM pushes ratings first and foremost, and both last.fm and rym push trending artists to their users).
Instead, because the value of music is zero (really, negative since the number of listens, streams, album purchases, etc can fail to recuperate the cost to make it ), the act of distributing music presents economic risk unless the release itself can be controlled by the investors through advertisement or paid promotion.
The premise that many folks miss here is the idea that Spotify is, at best, thinly interested in recommending music that is good for YOUR interests. Spotify is the music business, and specifically the pop music business, has long discovered that's it's much more economically expedient to force feed musical taste onto the public than it is to chase the whims of organic hit-making. Payola is as old as recorded music. Spotify recommends what Spotify wants it's users to listen to. They have all kinds of side deals and marketing deals with labels, they have cheaper costs/royalties on some tracks than others. Popular tracks cached in their CDNs are probably cheaper to recommend than long tail ones etc. They have strategic priorities like gaining on apple for podcasts, and therefore injecting allsorts of podcast recos in the UI wether you asked for that or not.
I have no idea if anyone is listening to them, though, because there doesn't seem to be any feedback system for the community playlists. That would be a useful addition, IMO. If TIDAL doesn't want to pay dedicated staff to curate playlists, they could at least make some way for the member-created playlists to get featured or gain reputation.
[0] https://www.deezer.com/fr/playlist/1588605745
This seems like the complaint of somebody who hasn't been using spotify very long. After a decade plus, I feel like my algorithm is a rich compost pile of all of my previous phases of music. Spotify is excellent at letting me broaden my horizons or jump down a rabbit hole from a random starting point, like a song I hear in a public space or commercial or something sent by a friend. Maybe the OP should keep their ears open to more sources of randomness from the outside world?
It probably helps that the strongest areas of my taste are relatively small or niche genres, like Scottish trad and Celtic (folk) rock. In those niches, similar-but-different is often distinctively different in actual experience. Sure, there's covers of the same song from time to time, but I actually do like enough of those not to be bothered, if they bring something new.