HN Explorer

Ask HN: Many quality articles don't go to the front page. What is the problem?

88 points by vladocar ↗ HN
I'm noticing lately many quality articles don't get any votes in the new links. What happened? Is HN grown so much that people submit many articles? Did you noticed the same problem?

44 comments

[ 2581 ms ] story [ 1699 ms ] thread
eru
I noticed similar problems. I guess it's the growth, and that an article only really has a few minutes to get its first few votes that will put it onto the front page.
lukeqsee
It could be growth. It could be a lot of things.

I postulate that the necessary quality level is greater now because of the sheer number of articles that get sifted through. What used to make the cut, no longer does. Or, it could be vice-versa.

ryanwaggoner
I think it's a function of there being too many items submitted every day, and not enough people checking the "new" page. During busy times of the day, if you don't get 5-10 votes within the first 10-15 mins, you're on the 2nd page of the "new" posts area, and you're never going to make it. I'm afraid the end result is that the only posts that will end up on the front-page are:

  1. Where the author (domain) is well-known
  2. Where the title is link-bait
  3. Where the submitter tries to game the system
All of these are bad for the future of HN. I really like the way reddit handles this, of putting a few new items at the top of the homepage and rotating them, so that more people see them and have the chance to upvote. The way things are going, HN is destined to have a front-page full of only linkbait, spam, and/or posts by rockstars.
makmanalp
Another idea may be to increase the amount of time that a new post is "boosted" into the main page list. It seems like new posts fall to the 2nd page really quick. So it might be fixable by tweaking parameters.
JangoSteve
Another obvious cause of submissions falling too quickly off the "new" posts page is that there is only room for the latest 30 posts. Seems like we could fit another 3-5 posts on the "new" page if we removed the posts that have already hit the front page. Of course, this is only a minor band-aid that doesn't solve the root problems, as you pointed out.
mhb
Getting dead ones off the page would yield another few.
JangoSteve
You can set "showdead" to false in your profile.
mattmcknight
The other way is for multiple people to post the same link, which adds to the total. Maybe if people started submitting more, or submissions from high karma types had more initial karma, it would have a cumulative effect.
mattmanser
There is a different solution to the reddit one, one that discourages block voting.

Perhaps there should be voting groups, essentially a cap on the population of voters.

Each time you hit a threshold, you randomly split the voters in two and then the articles you see only get the votes of the people in your group. So one site has multiple groups on the go at one time, all to stop the overwhelming nature of popularity drowning out the minority.

It stops block voting as all you can do is influence the group you are in rather than the community as a whole. There's a good chance on a big site you're not in the same group.

On the otherhand you could end up with groups that have wildly different front pages that don't suit your tastes. Perhaps to solve that there could even be something intelligent about it in that it switches you into groups with voting tendencies like your own, improving your perceived quality of the site.

Comments are a little troublesome, do you include them in the group voting or not? In one group a comment could be negative, while popular in another.

A random thought that occurred to me, I generally hate these regular meta threads of 'HN has got bad'.

Do you know what I saw today on HN? DHH and Joel Spolsky duking it out in the comments. Two of my favourite internet personalities arguing.

That rocked. HN is still the best place to go.

lincolnq
Wow. I tried to build a voting-based news-to-your-personal-taste site, but never did I think of this "groups" idea. I'm intrigued.

Idea: do it to HN as an A/B experiment: see how the content of the sites diverges. Actually the "/classic" url does this, sort of, by ranking articles not by total votes but by votes from users older than one year.

mattmanser
Now I think about it in the light of day it doesn't solve the problem of too many articles getting submitted.
nerfhammer
Yea it suggests really interesting experimental hypotheses: would the content in the different groups diverge or would it stay more or less the same? And would the degree of divergence change over time?

I wonder how strong the correlation would be.

amirmc
In some cases, the timing of the submission also matters.

e.g I've posted a few interesting articles that dropped off the new page

frossie
I am really curious as to the answer to this question:

Of the stories that got more than 10 votes, how many of those (first) 10 votes came from upvotes as opposed to multiple submission.

My theory is that it frequently takes multiple submissions to get stuff in the front page so that it can be upvoted, rather than through upvotes from New. In which case the best story in the world might not do well if only one person finds it.

10ren
Another approach is to prevent these - which is how pg seems to describe his role at HN these days, at least for (3) gaming the system - leaving the field clear for the good stories. The flag link often takes care of (2) link-bait (though most people complain rather than click it). Well-known (3) author/domain probably isn't too bad, if the brand is justified, and even seems to be countered at times, eg. Seth Godin doesn't come up much anymore.

The big problem I see is industry gossip, within which I include: the crunchpad drama; iPhone/iPad dramas; Zuckerman's integrity; Bill Gates' integrity; Steve Jobs' integrity; Larry Ellison's integrity; today's psychoanalysis of one of our own members; even the Ring of Dark Angels/AngelGate. Now, industry gossip is relevant to people in the industry. But it's not Hacker News (at least, not according to the guidelines.)

If there is a clear perception of what HN is, and enough people resolve to flag stories that don't fit that perception, then this problem goes away. We deserve the HN we get.

timdellinger
Sometimes I feel like a bit of a freeloader, since I never click on "new" in order to upvote the decent submissions.

The only suggestions that I have would be (1) to automatically load a handful of "new" submissions in a separate section of the main page (at the expense of destroying the current design and cluttering the page), or (2) giving out karma for upvoting new stories (which is ripe for exploitation by simply clicking randomly).

LaPingvino
Maybe considering an upvote on new while it's not on the frontpage yet as submitting the article (and thus getting karma according to the votes on the article) can help. Then upvoting good articles on new gets very attractive.
rblion
I think the HN community is becoming more and more diluted with people who are here for techcrunch-type gossip and articles and not actual thought provoking insights and research.
Zev
No more so than three years ago, when I started lurking. Or two and a half years ago, when I signed up. Or however long ago when I first commented.

People are interested in what other people are doing. Techcrunch happens to be a big resource in figuring this out. Thats not necessarily a bad thing in itself.

rblion
i like to know too. but only about the innovators and leaders, not so much the 100 me-too startups that try to make the same thing but with a different name.
Zev
Don't underestimate the usefulness of knowing what people are interested in at the moment. Social networking and the like may be gimmicky, but, they don't become popular for no reason.

// edit: A better "but, …" might be: "but, Facebook has an insane valuation for a reason: a lot of people seem to believe it can make a lot of money."

rblion
I never said anything about social networking being 'gimmicky'. It's fills a human need for belonging. The kinds of articles that get my attention explain how successful startups work or how others failed.
alain94040
My suggestion would be to randomly feature 5 new articles on the home page (probably below rank 20, so as not to pollute the top 10). This way, they get more visibility than by being only on the "new page", and get a chance to get the clicks they need.
ecaradec
I think that it's already the case : some articles are nofollow on HN homepage. They usually are 3 or 4 upvote at the bottom mixed with upvoted dofollow links.
bravura
Rich get richer phenomenon.

Not enough people check the new page. It's impossible to get an article on the frontpage unless like four or five people upvote it in the first few minutes after submission. This is extremely unlikely, unless you have a popular title or popular username.

Once something gets on the frontpage, it gets a LOT of upvotes, and the algorithm decays its weight so slowly that it is essentially a rich get richer phenomenon.

Possible solutions:

* Front page upvotes don't count as much, since there are more views. In particular, the "weight" of an article should be the probability that someone who views the title clicks on it, combined with the probability that someone who clicks the link decides to upvote it.

* Frontpage is stochastic. Instead of being fixed, you sample articles based upon their probability (or score). Each person gets a new frontpage, in order to actually explore which articles are good. (Exploration vs. exploitation)

tokenadult
I habitually check the new page right after checking the main page. I upvote stuff that I think is interesting to the community. Anyone else can do the same.
marilyn
I believe a flip side to this is that stories are staying on the front page longer. Not too long ago I recall that each day I opened HN for the first time, every story on the front page was brand new to me. Recently stories seem to stay on the front page for 2-3 days sometimes.
njharman
> stories seem to stay on the front page for 2-3 days sometimes

That is far too long. I'd like the front page to churn twice a day so it was all new in the "morning" and all new in the "afternoon".

epi0Bauqu
This has been going on for a long long time.
jasonkester
I've noticed this happening in the last 6 months. Articles of mine that spent the better part of a day on the front page of Reddit sail their way down the New page here over the course of 45 minutes, never to be seen again.

Funny, because a couple years back that's what would happen to anything you submitted to /r/programming. Now the noise has all moved here and it's actually easier to get Reddit to pay attention then HN.

spotirca
One small improvement that might help would be to hide the articles that one has already read, keeping the front page to 30 articles. That way, after you read some articles, there will be some fresh meat on the table.
jacquesm
It takes 4 points currently to make the front page, if your submission does not have those 4 points by the time it scrolls off the new page it will take a small miracle for it to be brought back.

Also notice that the 'new' page has a 'more' link at the bottom, you could do worse than to check page 2 and 3 as well to see if anything good fell through the cracks.

Voting up articles that have not made the front page past the third page of 'new' articles is unlikely to have much effect.

JeffL
Last time I tried to submit something and it didn't go anywhere, I checked the new page and it was full of multiple submissions from the same person. Personally, I think people should be limited to one submission per day, or else multiple submissions start to cost you karma, so people had better like them or else you get punished for Spamming.
reitzensteinm
I was stunned that my last submission, that the gog.com shutdown was a marketing ploy, received 0 upvotes. It was the latest development in the story that was voted #1 just two days prior:

http://news.ycombinator.com/item?id=1717426

It appears they're actually back now, so I've submitted it again:

http://news.ycombinator.com/item?id=1721598

reitzensteinm
I take it from the downvote and lack of general interest that the problem isn't that nobody saw it, it's that people genuinely didn't care. I'm surprised, but I stand corrected.
eneveu
Well... You simply linked to their front page.

This submission got more interest: http://news.ycombinator.com/item?id=1717172

I think such an article generates more interest because it puts everything in context, and explains GOG's strategy / hoax.

reitzensteinm
Thanks for the link - it puzzled me because I didn't realise the story had already done the rounds, and search didn't turn anything up. Makes total sense now.

The front page at the time had a post discussing what had happened - now it just links to the main site.

doki_pen
I know a lot of users use Google Reader and don't see the articles until they are a little stale. They still will up vote good articles. Perhaps the algorithm could be tweaked to make time less important then it is currently. I'm not sure how you could do that though..
3pt14159
Submissions should cost karma. Dynamically change the karma cost based on the time of day or expected number of visits. In theory good posts would be a net win for your karma and people would stop spamming the new section with useless articles.
winter_blue
I have an Greasemonkey script called HN Toolkit that lets you display the new post & top post side-by-side. It really helps you catch the good ones that aren't upvoted.
cinimod
Another suggestion: Adding a small section above the top10 with a random link from "new". And for that link, you need to click yes or no depending of if it is interesting.

What I like about this approach is the randomness in it where random people get random link. I also like the fact that by you aren't choosing between 30 links.. you only get one and you need to say if it's interesting or no. Also, I like the fact that it's on the front page.

So, the top 10 links could be the most yes-ed in a certain period, or the ratio yes/no ratio, or anything really.. You could also put more weight on a yes if you've got more point in HN (Simply because I usually trust their judgement.. for instance, they know when something has already been shown dozen of time)

Also, it feels a little bit more like a game to me.. each time I refresh I need to really participate to say if that special link is interesting or no.. :D