Great stuff, but please normalise to UTC in future. Everyone (reading this) knows their own offset from UTC and can probably makes sense of the data directly in that format. For anything else I have to go and lookup the offset between that timezone and mine.
Not everyone knows the difference between their timezone and UTC. I always end up looking up the difference, and it really doesn't matter to me what timezone people use, so long as they state it.
For a website with a US centric audience, I think it makes perfect sense to use a US timezone: The graphs match up well with the sleep/wake patterns in the EST timezone. This makes interpreting the data a bit easier, i.e you can easily tell that there is a morning rush, a lunch rush, etc. (although interestingly, the HN audience doesn't seem to have well-defined break hours as there is just one large peak per day).
My early submissions to HN all hit the front page and were upvoted quite a bit, sometimes reaching #2 overall. But in the past couple years, nothing I submit ever hits the front page, even if it gets more points initially than the submissions before. Not only that, it doesn't appear even on pages 2-20. I wonder if there is a way to check if I'm being suppressed somehow as a contributor on HN.
As someone who has looked into this area for Reddit and HN data, here are a few additional notes before everyone starts submitting their blog posts and marketing pieces to HN on weekday mornings:
1) After everything, the analysis is probabilistic. You are not guaranteed to get onto the front page just by submitting on at a high-probability time.
2) There are game-theoric implications; the more people submitting at a time (weekday mornings), the less likely people will see your post on /new before it is pushed off the first page, and it is highly unlikely to get upvotes once it has fallen off the top 30 slots. (hence the repost/second chance rules)
3) For obvious reasons, don't submit if there is important news occuring. (e.g. Apple/Google/Microsoft event)
4) Above all, if your submission does not get any upvotes, don't interpret it as your post being low quality. The median score of all submissions is 1-2 points regardless of time submitted. (http://i.imgur.com/SN5BuAJ.png)
The OP is impressively throrough regardless, especially since working with ranking data-over-time is much better than working with raw data.
To paraphrase the original article that inspired the OP, the Hacker News front page is not a meritocracy. There are optimizations that make your submission more likely to be upvoted (a simple one being a clickbait title). It is worth noting that this submission would have different behavior with the original title "You only need 3 votes to play, and other facts about the Hacker News frontpage." (and apparently it was just changed to that, and penalized?)
That's also why a good moderation team is necessary and important; to fix submissions which are not inherently good, and to boost submissions which are not inherently bad.
Also I think there is something to be said about the comments section of the post. I rarely up vote anything, and if I do it's usually an afterthought after I've read a good discussion in the comments. I've also caught myself on multiple occasions up-voting a submission after I made a comment on it, simply so that my comment would become more visible and be up-voted.
I think there are a lot of intricacies involved in these systems, and that they can be gamed for personal gain, ala much of reddit.
I frequently upvote submissions that have good comments, even if the article is not-so-good. There really should be two different upvotes, one for submissions the other for comments.
Also, your post will get weighted down if it contains certain words or is from a certain domain. I believe at one point 'NSA' got you penalised (due to the high number of NSA links coming in at one point in time), and Medium posts have a tougher time getting onto the front page (due to the lower signal:noise ratio I guess).
NSA got me penalized from 2013 onward if I said anything positive about them on about any security or IT forum. Evidence or accuracy didn't matter due to polarization. So, this is more true than you say. Less now but the effect is still there.
There's more than a bit of leftist elitism at work. E.g. I once made the mistake of submitting something from breitbart.com. It never appeared in 'new'.
When I'm logged in, it shows me that I've submitted 5 items. Examining my submissions anonymously only shows 4 items. Perhaps there are 5 if someone has 'showdead' on?
Putin's external propaganda site, rt.com, is OK, heaven forbid that Breitbart be afforded that same privilege.
I think the simpler reason is for a similar reasons as to why Medium get's penalised by the algo - links from Breitbart (and Medium) generate less useful discussions than other sites, thus they must 'work harder' to 'earn' their place on the front page.
Similar to the 'flame war detection' that flags posts with more comments than (up)votes.
HN is frustrating. I've had two blog posts hit #1 in r/programming recently. Neither got more than 2 points on HN.
I could be wrong but I think HN is more random than Reddit. Reddit has lots of randomness of course. But my gut says that escaping New on HN is harder and less predictable.
So submit them again, if they're interesting. That's both allowed and encouraged here.
Use your judgement though. It's a privilege to be able to do this. Three times over the course of a few days is probably fine. Four is probably too many.
I've also gotten "slow down, you're submitting too fast" when multiple days have passed. That might have been because I'd had a few comments get a bunch of downvotes? I comment a lot but almost never submit stories.
Speaking purely for myself, I hope to see them resubmitted. The post about reverse engineering Sublime's fuzzy matching algorithm is an overlooked gem, I think.
Nah, feel happy. People are agreeing that your content is good. That one got pulled because the automatic vote detector detected that a bunch of seemingly-artificial votes (ours) were being sent to it. As such, it certainly doesn't count as a resubmission. So really, nothing has changed.
Either way, thanks for contributing. I'm a gamedev too, so clicking on your submissions list instantly made me feel nostalgic. Tom Forsyth is a name I hadn't seen in awhile.
The "slow down, you're submitting too fast" thing is arbitrary and per-account. My account is specifically penalized, for example, and I have a far lower comment limit than pretty much everyone else. This was added without notification to me or appeal opportunity despite my e-mail being in my profile, and also means my comments are "demoted" below often punishingly-downvoted comments. So you'll see comments, then barely legible gray comments, then mine. I also cannot respond to deep threads because I burn my budget getting there, and then have to wait a couple days to continue a discussion. And that's just how my account is, now, even after conversing with them about it; Dan could not recall the specifics of why I was penalized but was resistant to removing it based upon a determination of my quality of character. From comments. On a forum.
My best guess is that there's a box on an account with an automatic and manual score adjustment, mine contains -100 or something, downvotes (real or synthetic) affect that timer that you see, but only 'real' downvotes trigger the somehow-still-exists-in-2016 'turn a comment gray' logic.
I'm not saying any of this to gripe, just that I doubt I'm the only one, I doubt that threshold is fully manual, and I doubt that the one for submitting links and the one for commenting are wildly divergent. A few days seems long, which is why I mention it. So perhaps you submitted something a million years ago that was quietly looked down upon and your account carries consequences; it wouldn't be the first time, since it was apparently one comment a million years ago that earned me my own status. And it matters so little nobody remembers.
Might be worth asking since they were more than willing to tell me once I asked, but it turned into a surreal "can we trust you to be a good boy if we remove it?" back-and-forth and I lost interest in attempting to satisfy Internet forum moderators with tales of personal development and growth. Good luck, if you do.
If a story has had significant attention in the last year or so, we kill reposts as duplicates. If not, a small number of reposts is ok. [...]
At times, each page of /new is only a very short amount of time, so it's easy for stuff to fly through and get lost.
Especially if nobody is looking at /new and voting. (seriously, everybody. You want to see more technical content? Vote for it as early as possible)
Stuff that's on mainstream sites is submitted by tons of people, even if it is already on spot 1 on the front page, individual blogs don't get that many chances sadly, and drowned out. (I feel weird reposting something the author submitted him-/herself the day before, but maybe we should do that more)
My experience is that it is much easier to get on the front page of reddit (specifically reddit/r/programming, aka Proggit). As I am writing this, my submission from earlier today there is at number 2 [1] - the same submission to HN got 4 points (which is actually good, but not enough to get to the front page). One reason I think is that the turnover at HN is much higher.
I suspect that HN is significantly less deterministic than other sites as to what content escapes New. I believe that a good content aggregator (HN, Reddit, Digg, etc) should be able to promote a given piece of quality content with a high degree of reliability. Some sites are better at this then others. My impression is that HN is less good. I would consider that a fault.
Obviously I don't have any hard evidence to anything I just said. Which is why I used words such as suspect, believe, and impression.
There is always going to be some variability. All content kneels before RNGesus. But some really good content never gets past two upvotes. Two! That could be a lot better. In my opinion.
HN is bigger than /r/programming. The subject matter is broader, and there are more submissions. Getting to the front page here is harder.
Second, new posts aren't even ranked before they get some upvotes, so they only appear on the "new" page. Not everyone looks at "new". The crowd that promote new stuff to the front page is not the same crowd that upvote stuff that's already on the front page. Some apparent randomness could come from this difference.
I've only made two submissions. By some odd coincidence both made it to #1, which has scared me off making any further submissions for fear of ruining my perfect score! :D
I also think that people shouldn't post small projects that don't belong to them - If a project/site author wants to be on HN, send them an email and suggest it. They should pick their "release" date, and be prepared for HN traffic.
Thanks, that's a better source than any I'd seen before. Have you thought about mentioning this in the FAQ? It seems to come up pretty often, and can be kind of confusing due to the timestamp manipulation.
That happened to my post on the number of Go positions
https://news.ycombinator.com/item?id=10950875
Same pattern; I posted late at night and late in the morning the submission time was reset.
Something that I've found to be very important in determining whether a submission is successful (here on HN and on reddit) is the first comment. If something has a handful of upvotes, people will often check the comments to see if the link is worth their time or not. If the first comment is "Wow, this is amazing! Why is this not on the front page?" you have a much better chance of making it than if the first comment is "This is blogspam. Move along." Once a post gets critical mass, and there is a conversation going on, people assume that the post is worth reading, and invest more time, which creates more upvotes. Of course, this isn't the best filtering mechanism since it is effectively the wisdom of the crowds.
Is there a breakdown based on the rough IP location of the people who submits articles?
I suspect that HN is also biased towards users in the western hemisphere; so much so that when people from East Asia (like me) post or comment, it is comparatively harder to accumulate points as our posting time is out of sync with when the majority of HN users will see them.
I suppose one way out of this is to post when my local time is past midnight but I haven't reached the point where I value my HN points that highly compared to daily life...<grin>.
>> I haven't reached the point where I value my HN points that highly compared to daily life...<grin>.
You will. :-)
But mostly I would stop worrying about it. I am always surprised how one comment seems to find the one person able to make an interesting answer, and that's basically why I still come back. One point, one comment. And everything else is gravy.
To a reasonable first approximation, everyone on HN is in Silicon Valley, although "Not US" is still substantial. Here is previous survey I submitted, not sure if there's any better source.
65 comments
[ 3.0 ms ] story [ 134 ms ] threadFor a website with a US centric audience, I think it makes perfect sense to use a US timezone: The graphs match up well with the sleep/wake patterns in the EST timezone. This makes interpreting the data a bit easier, i.e you can easily tell that there is a morning rush, a lunch rush, etc. (although interestingly, the HN audience doesn't seem to have well-defined break hours as there is just one large peak per day).
I'd expect them to match more PST/PDT (-4h), that's where the Silicon Valley and YCombinator are.
Source: see my past submissions.
1) After everything, the analysis is probabilistic. You are not guaranteed to get onto the front page just by submitting on at a high-probability time.
2) There are game-theoric implications; the more people submitting at a time (weekday mornings), the less likely people will see your post on /new before it is pushed off the first page, and it is highly unlikely to get upvotes once it has fallen off the top 30 slots. (hence the repost/second chance rules)
3) For obvious reasons, don't submit if there is important news occuring. (e.g. Apple/Google/Microsoft event)
4) Above all, if your submission does not get any upvotes, don't interpret it as your post being low quality. The median score of all submissions is 1-2 points regardless of time submitted. (http://i.imgur.com/SN5BuAJ.png)
The OP is impressively throrough regardless, especially since working with ranking data-over-time is much better than working with raw data.
That's also why a good moderation team is necessary and important; to fix submissions which are not inherently good, and to boost submissions which are not inherently bad.
I think there are a lot of intricacies involved in these systems, and that they can be gamed for personal gain, ala much of reddit.
There's more than a bit of leftist elitism at work. E.g. I once made the mistake of submitting something from breitbart.com. It never appeared in 'new'.
When I'm logged in, it shows me that I've submitted 5 items. Examining my submissions anonymously only shows 4 items. Perhaps there are 5 if someone has 'showdead' on?
Putin's external propaganda site, rt.com, is OK, heaven forbid that Breitbart be afforded that same privilege.
Similar to the 'flame war detection' that flags posts with more comments than (up)votes.
>here are a few additional notes before everyone starts submitting their blog posts and marketing pieces to HN on weekday mornings
I could be wrong but I think HN is more random than Reddit. Reddit has lots of randomness of course. But my gut says that escaping New on HN is harder and less predictable.
Use your judgement though. It's a privilege to be able to do this. Three times over the course of a few days is probably fine. Four is probably too many.
I've also gotten "slow down, you're submitting too fast" when multiple days have passed. That might have been because I'd had a few comments get a bunch of downvotes? I comment a lot but almost never submit stories.
https://blog.forrestthewoods.com/reverse-engineering-sublime...
https://gamedevdaily.io/four-ways-to-create-a-mesh-for-a-sph...
https://gamedevdaily.io/advanced-behavior-tree-structures-4b...
Speaking purely for myself, I hope to see them resubmitted. The post about reverse engineering Sublime's fuzzy matching algorithm is an overlooked gem, I think.
https://news.ycombinator.com/item?id=11515800
I like that it disappeared from the front page, or never showed up, because it means HN's voting ring detector is working very well. Kind of amusing.
I'm sure it'll pop up sometime, though.
How disappointing. :(
Either way, thanks for contributing. I'm a gamedev too, so clicking on your submissions list instantly made me feel nostalgic. Tom Forsyth is a name I hadn't seen in awhile.
(1) stop posting stuff anywhere
(2) stop going on the internet except for doing my job
It's raised my productivity and contentment a lot.
(Well, sometimes I break rule #1 to make comments like this. But that's about it.)
Only thing left to solve is http://i.imgur.com/95NZpPy.png and then I'll really be living the life.
My best guess is that there's a box on an account with an automatic and manual score adjustment, mine contains -100 or something, downvotes (real or synthetic) affect that timer that you see, but only 'real' downvotes trigger the somehow-still-exists-in-2016 'turn a comment gray' logic.
I'm not saying any of this to gripe, just that I doubt I'm the only one, I doubt that threshold is fully manual, and I doubt that the one for submitting links and the one for commenting are wildly divergent. A few days seems long, which is why I mention it. So perhaps you submitted something a million years ago that was quietly looked down upon and your account carries consequences; it wouldn't be the first time, since it was apparently one comment a million years ago that earned me my own status. And it matters so little nobody remembers.
Might be worth asking since they were more than willing to tell me once I asked, but it turned into a surreal "can we trust you to be a good boy if we remove it?" back-and-forth and I lost interest in attempting to satisfy Internet forum moderators with tales of personal development and growth. Good luck, if you do.
Are reposts ok?
If a story has had significant attention in the last year or so, we kill reposts as duplicates. If not, a small number of reposts is ok. [...]
At times, each page of /new is only a very short amount of time, so it's easy for stuff to fly through and get lost.
Especially if nobody is looking at /new and voting. (seriously, everybody. You want to see more technical content? Vote for it as early as possible)
Stuff that's on mainstream sites is submitted by tons of people, even if it is already on spot 1 on the front page, individual blogs don't get that many chances sadly, and drowned out. (I feel weird reposting something the author submitted him-/herself the day before, but maybe we should do that more)
[1] https://henrikwarne.com/2016/04/17/more-good-programming-quo...
Obviously I don't have any hard evidence to anything I just said. Which is why I used words such as suspect, believe, and impression.
There is always going to be some variability. All content kneels before RNGesus. But some really good content never gets past two upvotes. Two! That could be a lot better. In my opinion.
Second, new posts aren't even ranked before they get some upvotes, so they only appear on the "new" page. Not everyone looks at "new". The crowd that promote new stuff to the front page is not the same crowd that upvote stuff that's already on the front page. Some apparent randomness could come from this difference.
I also think that people shouldn't post small projects that don't belong to them - If a project/site author wants to be on HN, send them an email and suggest it. They should pick their "release" date, and be prepared for HN traffic.
My most upvoted submission was posted on a weekday night - the timestamp changed in the morning then it got several hundred upvotes.
A: Page 2 of Hacker News.
I suspect that HN is also biased towards users in the western hemisphere; so much so that when people from East Asia (like me) post or comment, it is comparatively harder to accumulate points as our posting time is out of sync with when the majority of HN users will see them.
I suppose one way out of this is to post when my local time is past midnight but I haven't reached the point where I value my HN points that highly compared to daily life...<grin>.
You will. :-)
But mostly I would stop worrying about it. I am always surprised how one comment seems to find the one person able to make an interesting answer, and that's basically why I still come back. One point, one comment. And everything else is gravy.
https://news.ycombinator.com/item?id=6583918
https://www.google.com.au/trends/explore#q=hacker%20news
Not sure how accurate Alexa is. Generally between India, China, Australia and Indonesia they must pull a fair size audience.