Are HN submissions dominated by a small elite of users?
I thought it would be easy to find out how karma is distributed among users using the HNsearch API.
Unfortunately, it's not that easy because this API is not optimized for fetching massive amounts of data. What I could do was looking at the 1000 users with the most karma. I separated those into ten chunks and looked at the total sum of Karma per chunk of 100 users. This is the result:
http://www.chartgo.com/share.do?id=6cca3d09aa
(Data: [ (0, 2941707), (1, 1314760), (2, 936226), (3, 745944), (4, 636533), (5, 557441), (6, 493996), (7, 449999), (8, 413963), (9, 384491)] )
So as you can see the 1st chunk is pretty dominant but then it kind of flattens out. So are HN submissions dominated by a small elite? It obviously depends on your definitions, but I guess not.
It would be interesting to have better data available. Did I miss a better way to query the API? Here's the Python script which I used to query the API: https://gist.github.com/johannes-gehrs/77a92284e3509f2ae960
27 comments
[ 4.2 ms ] story [ 89.1 ms ] thread"Dominated" is a flawed qualifier since Karma is not a zero-sum game.
For comments, at least, there is a clear bias where comments from users with higher karma tend to stay at the top of the comment page longer, regardless of the votes for other comments. This is intentional and definitely contributes to the effect
If you're talking about comment karma, clearly yes. I could write a comment consisting of randomly chosen names from the Chicago phone book and someone would upvote it. Name recognition has that effect, as does a small group of people who seem to follow my comments deliberately (as I do for the list of people in my profile).
Not that I disagree with this algorithm, I actually agree. Just saying that I think the position of the comment is more important than name recognition in this case.
Net - domination is too strong a word. Dominated might imply 20% of the karma in 20 users, and this clearly isn't the case. Even defining the Elite as 100 doesn't get us this.
I don't think it typically matters whether the submitter has name recognition or karma, but rather how many friends and colleagues the person can rally to their cause. All it takes is about 10 points in the first hour to get to the front page and then it can "ride the wave" from there. This is an easy threshold for people with large professional networks, companies, with friends in SV et al. Many people here know each other. Yes, there are some technical counter-measures, but there is only so much that can be done.
This is not to say that a user acting alone has no chance of reaching the home page. However, after a few years of frequenting this site, I believe that most of the posts hitting the front page have had a boost such as I described above.
My personal experience anecdote is that, over several years (also considering prior accounts), working as a "loner", that I've never been able to get more than a few meager votes on a submission. Even if they were highly on topic, even if I got up at 6am Pacific, supposedly the best time to do it. Maybe I'm unlucky or my submissions suck, but I rarely bother any more.
I could be mistaken, but the hypothesis fits so far.
1. Set up IRC channel. 2. Wait for friends to arrive. 3. When there's ten, post new content to social media. 4. Post link to the IRC. 5. Have you and 1-2 close friends downvote everything else.
All of the algorithms I've seen so far are susceptible to this.
I do not engage in this practice for many reasons, some ethical and some practical.
On an ethical level, it's toying with people's faith in society; some trolls argue successfully that this faith is misplaced and should be shaken; I counterargue that without strong signaling to that effect, it becomes vandalism.
The practical is that anyone who trolls around the sites can see who is engaging in this practice and rapidly realizes they're behaving badly. I remember the crucifixion of davisreis666 on Reddit as an example.
(oh, and I submitted a link to your newsletter. maybe we can get it on the front page!)