Ask HN: Why do recommender systems not seem good?

They are not optimized for what you enjoy, or even want.

A passable analogy: you buy a car and get hassled, often hard-sold for a pre-paid maintenance package, tire insurance, financing insurance, undercoating, bla bla. You don't want any of it but it's what they push the hardest.

They work well if there are lots of people like you (ie you are a normie).

All machine learning algorithms struggle at the edges — they’re very good at predicting aggregate behavior.

If you have eclectic tastes there’s probably not enough data on your demographic.

I have a theory that some people's likes are based on things that are too subtle, intangible, or unrepeatable to identify, predict, or data mine. At least that's what I've figured is why recommendation systems don't work for me and why I run into so many dead ends when I try to use clear aspects of what I like to find other likes.

Excellent question. All the streaming providers only seem to suggest movies from my generation and absolutely nothing new. It feels like they have created an ultra generic advertising profile for me and only use that to recommend content. It's incredibly depressing. Youtube is slightly better but still extremely siloed. No amount of "algorithm training" seems to help.

You buy a $1000 scooter from Amazon and then it keeps recommending to you other good scooters for months, even after the return window for the first one is closed.

Yeah, you'd expect more from ML at this point. I wonder how much of ML research actually gets utilized in industry.

Scale. Providing accurate recommendation algorithms for thousands+++ of people across thousands+++ of data items is surprisingly expensive in compute and electricity. For any one user, sure you can do whatever you like. When you divide your resources across your userbase the prices get larger and larger.

Because the recommendations aren't for you - they are what generates the most amount of ad revenue for them, which roughly correlates to the amount of money advertisers are willing to spend to reach people like you. That is in no way a guarantee that the recommendations will be good for you, because the incentives are not aligned.

I think a lot of recommendation system take only the positive signals into account but don't take the negative signals that seriously.

So If I like and disliked 10 movies just don't show me movies from users who also liked these. First, filter or downgrade all users who liked what I disliked and then create my recommendations.

Often they just use "people who watched X also watch Y", which actually selects for the most popular stuff instead of the most similar.

Because recommendation is psychology-complete.

It is extremely hard to predict human behavior beyond simple schemes such as most popular items (or most similar items to those you’ve seen before).

(bio: six years of xp in a leading recommendation company)

I think that the issue is that it's always assumed that similar things are a better recommendation. The issue is at least for myself and other people I've talked to about that a lot of times we're looking for something different from the last thing. For example if I spend a week listening to vintage surf rock the recommendations that could be given might be 60s pop, or more surf rock. But what actually I wanted was to listen to experimental jazz with a retro funk twist on it. How could they anticipate that? Talk to anyone who's deep into some art form, movies, tv, music, and you'll see recommendations given that maintain vibes more than just similar things.

It's weird but tiktok is the only one that seems to do a good job. Seriously. The tiktok recommendation algorithm is so good. And then for ads, instagram seems to be the only one that does a good job. I regularly get ads for things I actually want from instagram, never from any other platform.

> YouTube seems to have a massive recency bias

This is likely intentional to encourage more content creation. Competing with two decades of content is almost impossible, so they make them compete with just 2 weeks of content.

I think it's partly just a reflection of how complex and poorly understood taste is.

Do you think these are optimized for what YOU want or for what the company wants.

YouTube: Is that recommendation for good content or for the highest value content you will consume?

Netflix: The more you use it, the less they make. There is a perverse incentive to put just enough good content in front of you to stay subscribed but not use it more.

Amazon: They dont give a fuck what you buy, the sellers are now in a race to the bottom and that business pays for it self. AWS makes all the money.

Find the perverse incentive and optimize for that.

Mostly conjecture on my part, and an anecdote: for some things, I have very specific and niche tastes that come in the form of things that are difficult to label.

Example: rap music and hiphop. For the most part, I don't enjoy it that much. There are a few things though that will make a track palatable to me (or instantly turn me off despite anything else positive about it):

    - Sentimentality or romance in the lyrics
    - Backing track or samples that are harmonically interesting
    - No egregious sexism, misogyny, glorifying of violence, thug/gangbanger culture, etc
    - Beats featuring stereotypical trap hi hats kind of annoy me
 

I've enjoyed tracks like Deja Vu by Post Malone, or Lucid Dreams by Juice WRLD. Browsing the rest of their discography consistently disappoints me though, because tracks like these are few and far between.

The way I assume recommendation systems are traditionally designed does not account for this. It sees me listen to these tracks, and thinks I'll probably like something by similar artists or the same artists. As far as I'm aware, Spotify's recommendation system is not aware of things like tempo, meter, tonality, themes of the lyrics, harmony, etc. and so there's no way it can pick tracks like this out from the crowd.

And why would they bother? Those are all much more technically difficult things to implement than forming correlations between IDs in a database.

Have you used the tiktok one? It's too good for my liking. It can make weird inferences that no other algo can (this is actually why I deleted tiktok, far too addictively good)

Have some new ideas for that. Have a Ph.D. in math, and derived some math for that, wrote it out with theorems and proofs in TeX. Should do better than current AI. Have Web site code, running as intended, for that. Collecting data.

You and your enjoyment is no longer center to the recommendations. Ads/money, engagement, and your time are the only metrics that matter.

Results these days seem worse then the old days of Altavista and Lycos.

Would LLMs change anything?

Do you use ad-blocking technology?

I have found YouTube to be good, but it gives you exactly what you ask for. Its like a yes-man for better or worse (this seems to be extended to people who are making videos now, too). But, once you are aware of that, it is not so bad, you just have to search for and like the right stuff. I like new, low view count videos like people doing $hobby with no commentary. That is mostly what I get now mixed in with some other things that I like.

Agenda. They want you to consume what they want. Have you tried using search on YouTube? First three results somewhat related to what you want, followed by a section of shorts and the rest is complete bullshit.

...and I'll never stop saying this: they have some sort of monetized recommendation system in place. Can't prove it but I can see it working almost every week. A video of a big company, "celebrity" or TV channel that I'd never watch, find it's way in my feed.

Not an expert, but as a learning experience I wrote a board game recommender years ago based on data from boardgamegeek.

There were a lot of little things that added up:

  1. Everyone interprets the 1.0 - 10.0 rating scale differently.
  2. Most users just rate the same, universally known games.
  3. For the other users, the games they've played are usually really different.  It's a sparse matrix.

Every attempt at game-to-game analysis flopped. User-to-user analysis seemed to work better.

I managed to find a few dozen similar users. Found some hidden gems by going through their pages manually. Fewer than I would have hoped though.

Because to make good, personalized recommendations they need to know what you don't know, and there isn't a good source of information about it.

For example, if you have not seen The Shawshank Redemption, chances are you will like it. It's #1 among IMDB top 250 list. But a recommender does not know if you've seen it. If you've seen it already, it's a bad recommendation.

So the same recommendation for the same person can be good today and bad tomorrow, depending on something recommender engine does not see. That makes it very difficult to tune and measure performance.