Negative "Sentiment" aka a black box made a frownie face.
Real discourse tends to be critical. If you want sloppy trade press, read Apple Insider or Business Insider, or maybe watch a slop tech creator like Linus Tech Tips.
Negative posts that I post tend to do better than neutral or positive ones. I have a classifier that judges titles on "most likely to get upvoted" for which "Richard Stallman is Dead" is the optimal title, and another that judges on "likely to have a comments/vote ratio > 0.5" [1]. The first one is a crummy model in terms of ROC, the second is pretty good and favors things that are clickbaity, about the battle of the sexes, and oddly, about cars.
But that 35 as an average score is hard for me to believe at first, I mean, the median HN post gets no votes, last time I looked the mean was around 8 or so. What is he sampling from?
Sent you a message to the mail in your bio but I thought I also leave this here: Again, thank you for your comments. I went down a rabbit hole reading your essays Hacker News For Hackers and Classifying Hacker News Titles.
I've also been building an RSS reader with integrated HN success prediction – essentially trying to surface which articles from my feed are most likely to resonate on HN before I submit them. Your research directly informed several decisions, so I wanted to share what I've learned and ask for your insights.
V1 DistilBERT baseline: Started with a fine-tuned DistilBERT on ~15k HN titles. Achieved ROC AUC of 0.77, which felt promising. Clean architecture, simple training loop.
V2 Added complexity: Switched to RoBERTa + TF-IDF ensemble, thinking more features = better. ROC AUC dropped to 0.70. Precision suffered. Calibration was poor (ECE 0.11).
V3 Stacking meta-learner: Added LightGBM to combine RoBERTa + TF-IDF + 30 engineered features (domain hit rates, author patterns, temporal encoding, word-level stats). The model was "cheating" by memorizing historical domain/author success rates from training data:
- domain_hit_rate: 79.9% importance
- roberta_prob: only 18.6%
V4 – Return to simplicity: Pure RoBERTa, no stacking. Added isotonic calibration (your probability clustering problem, solved!). Current performance:
- ROC AUC: 0.692
- ECE: 0.009 (excellent calibration)
- Optimal threshold: 0.295 (not 0.5 – exactly as you documented)
What worked for me:
Isotonic calibration: Your observation that most predictions cluster below 0.2 was spot-on. Isotonic regression produces well-distributed, meaningful probabilities.
Aggressive threshold lowering: At ~10% base rate (posts hitting 100+ points), 0.5 threshold catches almost nothing useful.
Pure transformer, no feature engineering: Contrary to intuition, adding TF-IDF and engineered features mostly added noise. The transformer handles the semantics well on its own.
What didn't worked for me:
Focal Loss: Made the model too conservative (760 FN vs 219 FP)
Domain/author features: Feature leakage, didn't generaliz
Stacking: Added complexity without improving generalization
Your essay mentions achieving similar ROC AUC with logistic regression on bag-of-words features. A few things I'm curious about:
Do you still maintain this system? Has your approach evolved since 2017?
What was your experience with full-content vs title-only classification? I'm title-only currently, which has obvious limits.
Any insights on the non-stationarity problem? Topic drift (Apple launches, security panics) seems like a persistent challenge.
What made you choose logistic regression over neural approaches at the time? The simplicity seems to have served you well.
This is the third link off the HN Front Page that yields the following error in Firefox:
Websites prove their identity via certificates. Firefox does not trust this site because it uses a certificate that is not valid for philippdubach.com. The certificate is only valid for the following names: cloudflare-ech.com, *.cloudflare-ech.com
> It's human nature: https://en.wikipedia.org/wiki/Negativity_bias. Everyone does it, but we perceive other people as doing it more than we do, which is itself a variation of the bias.
You can even see it in the title of the OP, in the word "overwhelmingly". That's excessive: the negative bias is noticeable, but if you look closely, it's not overwhelming. (To make up some numbers, it's more like 60-40, not 90-10.)
However, it often feels as if it is overwhelming; in fact, one or two datapoints, plus negativity bias, are enough to create just such a feeling. The feeling gets expressed in ways that trigger similar feelings in other people, so we end up with a positive* feedback loop.
The interesting question is, what factors mitigate this? how do we dampen negativity bias? or, how do we get negative feedback into our positive feedback loop of negative affect? That must also be happening all the time, or we'd be in a "war of all against all", which isn't the case, though (again) it may feel like it.
* ['positive' in the sense of increasing; a positive loop of negative affect!]
This was a really cleaver post to create the most comments of the sentiment mentioned. I am really impressed how well it is working and would like to know more.
Could you clarify what do you mean by "points" ("score" in your pre-print)?
Also, what's your data source?
"This study uses publicly available data from Hacker News." is not really a data source.
Also also, you missed the largest, most important effect that skews the votes on this site. But you can definitely find it on the dataset you got, that'd be a very interesting disclosure!
Hint: Immediately after posting this comment, it went to the bottom and "downvotes" magically started to come in ;).
I've seen the same with comments (both negative sentiment and shorter length). Short, snarky, negative comments [0] normally get a much better response than well-reasoned, longer-form comments.
Not that karam matters on HN but I have been disappointed to see longer comments that I put a lot of effort into get ignored while short, pithy comments get way more upvotes/replies. I've spent literally over an hour on some detailed comments that didn't even get a reply from the original person asking a question and likewise had comments I fired off with near-0 thought that "blow up". It's frustrating that better content is not always rewarded.
This is spot on and has really reduced my willpower to post, tbh.
Like begets like. A glib and snarky comment gets an emotional response, leading to quick, emotional votes. A nuanced, thoughtful comment gets the reader to think, but that's rarely conducive to upvotes unless they were already in agreement.
Over time everyone is Pavlovian-conditioned by the dopamine hit of upvotes to stick with the glib and snarky comments.
The whole upvote system is just a slow-acting poison that inevitably destroys any online community. HN has fared better than most, thanks to great moderation, but it won't resist forever.
I'd attribute at least some of that to HN users being weighted toward engineering-ish jobs. "Major Outage at US-East-1" news is something we are paid to pay far more attention to than "All is Well at US-East-1".
We self filter for negative responses because negative content is functionally interactive whereas positive content is functionally complete. Agreement is silent, users keep scrolling if they agree. Disagreement demands expression.
Just a theory I don’t have data to back this up.
145 comments
[ 3.0 ms ] story [ 78.4 ms ] threadReal discourse tends to be critical. If you want sloppy trade press, read Apple Insider or Business Insider, or maybe watch a slop tech creator like Linus Tech Tips.
But that 35 as an average score is hard for me to believe at first, I mean, the median HN post gets no votes, last time I looked the mean was around 8 or so. What is he sampling from?
[1] comments/votes = 0.5 is close to the mean
I've also been building an RSS reader with integrated HN success prediction – essentially trying to surface which articles from my feed are most likely to resonate on HN before I submit them. Your research directly informed several decisions, so I wanted to share what I've learned and ask for your insights.
V1 DistilBERT baseline: Started with a fine-tuned DistilBERT on ~15k HN titles. Achieved ROC AUC of 0.77, which felt promising. Clean architecture, simple training loop. V2 Added complexity: Switched to RoBERTa + TF-IDF ensemble, thinking more features = better. ROC AUC dropped to 0.70. Precision suffered. Calibration was poor (ECE 0.11). V3 Stacking meta-learner: Added LightGBM to combine RoBERTa + TF-IDF + 30 engineered features (domain hit rates, author patterns, temporal encoding, word-level stats). The model was "cheating" by memorizing historical domain/author success rates from training data: - domain_hit_rate: 79.9% importance - roberta_prob: only 18.6% V4 – Return to simplicity: Pure RoBERTa, no stacking. Added isotonic calibration (your probability clustering problem, solved!). Current performance: - ROC AUC: 0.692 - ECE: 0.009 (excellent calibration) - Optimal threshold: 0.295 (not 0.5 – exactly as you documented)
What worked for me: Isotonic calibration: Your observation that most predictions cluster below 0.2 was spot-on. Isotonic regression produces well-distributed, meaningful probabilities. Aggressive threshold lowering: At ~10% base rate (posts hitting 100+ points), 0.5 threshold catches almost nothing useful. Pure transformer, no feature engineering: Contrary to intuition, adding TF-IDF and engineered features mostly added noise. The transformer handles the semantics well on its own.
What didn't worked for me: Focal Loss: Made the model too conservative (760 FN vs 219 FP) Domain/author features: Feature leakage, didn't generaliz Stacking: Added complexity without improving generalization
Your essay mentions achieving similar ROC AUC with logistic regression on bag-of-words features. A few things I'm curious about: Do you still maintain this system? Has your approach evolved since 2017? What was your experience with full-content vs title-only classification? I'm title-only currently, which has obvious limits. Any insights on the non-stationarity problem? Topic drift (Apple launches, security panics) seems like a persistent challenge. What made you choose logistic regression over neural approaches at the time? The simplicity seems to have served you well.
As an ESL person one of the first internet-related terms I learned was "flamewar".
EDIT: ESL -> English as a Second Language
This is the third link off the HN Front Page that yields the following error in Firefox:
Websites prove their identity via certificates. Firefox does not trust this site because it uses a certificate that is not valid for philippdubach.com. The certificate is only valid for the following names: cloudflare-ech.com, *.cloudflare-ech.com
Error code: SSL_ERROR_BAD_CERT_DOMAIN
> It's human nature: https://en.wikipedia.org/wiki/Negativity_bias. Everyone does it, but we perceive other people as doing it more than we do, which is itself a variation of the bias. You can even see it in the title of the OP, in the word "overwhelmingly". That's excessive: the negative bias is noticeable, but if you look closely, it's not overwhelming. (To make up some numbers, it's more like 60-40, not 90-10.)
However, it often feels as if it is overwhelming; in fact, one or two datapoints, plus negativity bias, are enough to create just such a feeling. The feeling gets expressed in ways that trigger similar feelings in other people, so we end up with a positive* feedback loop.
The interesting question is, what factors mitigate this? how do we dampen negativity bias? or, how do we get negative feedback into our positive feedback loop of negative affect? That must also be happening all the time, or we'd be in a "war of all against all", which isn't the case, though (again) it may feel like it.
* ['positive' in the sense of increasing; a positive loop of negative affect!]
https://news.ycombinator.com/item?id=40430263
But posting something positive and getting slammed in the comments? That's depressing. So the barrier to posting something positive seems higher.
Could you clarify what do you mean by "points" ("score" in your pre-print)?
Also, what's your data source?
"This study uses publicly available data from Hacker News." is not really a data source.
Also also, you missed the largest, most important effect that skews the votes on this site. But you can definitely find it on the dataset you got, that'd be a very interesting disclosure!
Hint: Immediately after posting this comment, it went to the bottom and "downvotes" magically started to come in ;).
Not that karam matters on HN but I have been disappointed to see longer comments that I put a lot of effort into get ignored while short, pithy comments get way more upvotes/replies. I've spent literally over an hour on some detailed comments that didn't even get a reply from the original person asking a question and likewise had comments I fired off with near-0 thought that "blow up". It's frustrating that better content is not always rewarded.
[0] Something I'm guilty of
Like begets like. A glib and snarky comment gets an emotional response, leading to quick, emotional votes. A nuanced, thoughtful comment gets the reader to think, but that's rarely conducive to upvotes unless they were already in agreement.
Over time everyone is Pavlovian-conditioned by the dopamine hit of upvotes to stick with the glib and snarky comments.
The whole upvote system is just a slow-acting poison that inevitably destroys any online community. HN has fared better than most, thanks to great moderation, but it won't resist forever.
My usual journey: I visit the comment section and then look for the first top comment that criticizes the core thesis of the article.