Tell HN: I curate HN stories which didn't reach the front page
I'd like to introduce, and ask feedback for, my newsletter Interesting Things [1][2].
It's a response to the notion that many interesting stories posted to HN flies past and are not seen. Maybe they could, and should, be surfaced again somehow. A newsletter seemed like a good way to do it.
I experimented by trawling through the 'newest' section of HN. The first few times were mind-numbing. Gosh, there's so much noise in there! . But eventually I became more selective and efficient and it became repeatable.
Then I evolved it and added stories from reddit, newsletters that I subscribe to, and some other places. Whilst a portion of the stories are still from HN, it's no longer only from there.
The criteria is 'What I find interesting' but, since I'm a typical HN reader, the interest profile ends up being similar. It's mainly tech but with splashes of startups, science, productivity, etc. I omit politics. I also try to omit product/press releases, big corp stuff, and stories that would be covered by the mainstream tech press.
There is little overlap with other Hacker-News-based newsletters and digests. This is intentional. They cover the top stories from HN. Mine does not. (Maybe there might be one or two 'tier 1' stories, but the majority are not.)
So anyway ... if you'd like to read interesting stuff (whether or not it was unnoticed by HN) please have a look at Interesting Things. I hope you guys like it and find it ... well ... interesting.
I'm also looking for criticism, feedback and suggestions for improvement. Some sample questions: Do you like the one-two liner blurbs? Are the blurbs useful? What do you think of the distribution of topics? Are there areas where you'd like to see more (or less) coverage? Please let me know. I'm happy to talk.
Thanks for reading!
[1] https://bengtan.com/interesting-things
[2] https://bengtan.com/newsletter/sample (redirects to the latest public edition)
62 comments
[ 2.8 ms ] story [ 125 ms ] thread[1] https://bengtan.com/interesting-things
[2] https://bengtan.com/newsletter/sample (redirects to the latest public edition)
Considering the source of your content, and lack of attribution; now it's looking shifty.
I'm not.
It's only delayed when it gets posted to the web archive. Newsletter subscribers get it straight away.
And ... (Full transparency mode) it's to encourage readers to subscribe.
> Considering the source of your content, and lack of attribution; now it's looking shifty.
Please see comments above, particularly https://news.ycombinator.com/item?id=27493014.
Here's a question: If this isn't to be monetized, and this isn't about lead capture; why are you incentivizing subscriptions with current content instead of adding it to your feed?
When I get enough numbers, I'll include a link to a malicious website. Then people click on it and it'll download malware onto their computer. That will seed my botnet and then I can do bad things with it.
Of course when that power of a solid audience is actually there, then the more profit seeking intentions may come in.
Still, there’s no way to tell what path the person will go down. I don’t think assuming bad faith in your last paragraph is needed.
A trope on HN is acting holier than thou or rationalizing own behavior. IE piracy of media or ad blocking scorched earth style without true principled reason like malware or tracking.
The biggest I have found is the person likely doesn’t make their paycheck now or in the recent past from super principled pious-like work. Or they did make their money that way and are now changed.
Like The Social Dilemma film. Almost entirely filled with multi millionaires and two billionaires who made at least a chunk of their fortunes from social media. Money they are most certainly keeping and power/influence gained they are most certainly using too — I mean they are on this trending big documentary talking about helping the common man vs big social media/tech while not being the common man because of money from said industry.
Later on, the story reaches the front page on HN but it's a new HN link. So if I include the 'original' HN link, it's not useful to the reader, and may even mislead them into thinking there was insignificant HN discussion.
For example, I linked this story:
[Web Applications from the Future: A Database in the Browser](https://stopa.io/post/279)
in early May but it only recently garnered significant discussion a month later in early June (https://news.ycombinator.com/item?id=27424496).
So I'm not entirely sure how to resolve this.
In the case of your example https://hn.algolia.com/?query=Web%20Applications%20from%20th...
But not sure if that’s a good idea or not. Just suggesting it as one alternative to consider
I'm gonna think on how to solve this conundrum in a way that satisfies everyone. This is one data point I'll think about. Thanks for bringing it up.
1. It's interesting to see that a story has been submitted many times, for instance. This can be seen by clicking on the domain name in brackets after the story
E.g., a few days ago I posted a link to Heidi Howard & Ittai Abraham's 'Raft does not Guarantee Liveness in the face of Network Faults'
It was the fourth time this 6-month-old link had been posted, each time attracting upvotes but not enough to get on the front page. (I think it usually takes about five upvotes for a story still on the first page of 'new').
https://news.ycombinator.com/from?site=decentralizedthoughts...
2. Given that your newsletter mostly contains links that are less than a week old, the discussion link will still be live. You seem to think discussion is most likely to happen on a resubmitted link: I'm guessing your newsletter's readership is unlikely to be large enough to get the regular kind of front-page HN discussion going for many of the stories in your newsletter, but the opportunity to comment can still have value even if only a couple of comments are posted.
3. If a story is well-written, I generally like to glance at other things the author has written, and looking at stories that have been upvoted here on HN is often a good place to start.
It uses a giant Bloom filter with every article ever submitted to preserve user privacy.
If you normally enjoy being linked back to HN discussion, maybe you will like it!
https://github.com/jstrieb/hackernews-button
Direct string comparison of the current URL to previously submitted ones doesn't work because there are many ways for two identical web pages to have different URLs. For example, the URL fragments can differ (the part after the "#" that may or may not be present). Also there can be tracking parameters (often—but not necessarily—prefixed with "utm_"), which don't change anything about the page. But the URL parameters can't be entirely disregarded because sometimes sites, forums in particular, rely on them – consider pages that use an "?id=..." parameter for different pages. Thus some parameters should be removed, but some shouldn't. The same website having different domains (or domains that change over time) further complicates the situation.
My solution was to "canonicalize" URLs by transforming them into a simplified form using some pretty rough heuristics for common sources of noise. The Python code to do that is here: https://github.com/jstrieb/hackernews-button/blob/master/can...
All of this to say that even though I've used my extension for months and have been quite happy, there will inevitably be false negatives.
[0] https://en.wikipedia.org/wiki/Canonical_link_element
This only applies to the latest issue which is subscriber-only for 1 week. If you subscribe, the latest issue is emailed to you a few minutes later.
Earlier issues are publicly available in the archives.
> 11 Jun 2021 - Interesting Things #8 — Meta-/beta-stability (available only for subscriber until ??/??)
> 4 Jun 2021 - Interesting Things #7 — Don't read everything
> 28 May 2021 - Interesting Things #6 — Esoteric languages and sleep sort
I guess most people click in the first link and if it doesn't work assume that all the other will not work.
The problem though is that it is hard to find users you want to subscribe to. You need to inspect their past submissions to see if the signal-to-noise ratio of that user is high enough for you. If you oversubscribe to too many users then your subscriptions will be full of noise.
In my hobby project https://linklonk.com I want to solve this problem of discovering users and feeds that have a high signal-to-noise ratio for you.
It works similar to HN - users submit links and upvote/downvote them. But the way your ratings are used is different.
When you upvote something - you get connected to other users that upvoted the same article/post. The more shared upvotes you have with someone => the stronger you are connected to them => the more weight their other upvotes have for you.
If you downvote something - your connections to those who upvoted that item become weaker, their other upvoted have less weight for you.
Your connections to other users capture how useful their previous recommendations have been for you. And over time you get connected to the best content curators for you. All this done automatically as you rate content. You do not need to manually manage who to subscribe to/unsubscribe from.
The above mechanism works not only for users but also for RSS feeds that posted the content you upvoted. This way LinkLonk works as an RSS reader that prioritizes based on your content ratings.
Also, sometimes a story reaches the front page after it appears in my newsletter because it got re-submitted. Correlation? Don't know.
* the article's quality (denoted by x) and
* user_a's click happiness.
P(upvote_a | seen_a) = exp(a + λₐ x) / (1 + exp(a + λₐ x))
Both a and λₐ are unknown parameters that can be estimated from past data given some prior over x, say centered at 0 with a high variance.
One can grant a new articles a 2nd look if the posterior probability of x crossing a gate keeping threshold is sufficiently high.
Is supposed to be substituted when it is send as an email?
Yes.
I copy-and-paste directly from the newsletter content into the web archives. That's how this artefact gets transferred across.
I never sign up of extra emails... But I do occasionally add stuff to my RSS reader..
Which is what I read when HN frontpage has nothing interesting.
Amongst other things, I'd like to build up a mailing list. But I realise newsletters don't work for everyone, which is why I also post to a freely available web archive.
> Which is what I read when HN frontpage has nothing interesting.
Well, the next time you have nothing interesting to read, feel free to peruse my newsletter archives. :)
Just FYI that's why I (and I assume the OP) don't sub to newsletters.
Exactly, whenever I see "newsletter" I think spam.
Because clearly your content isn't good enough that people will come back on their own.
Newsletters can't be Show HNs. Please read the rules—they mention this explicitly: https://news.ycombinator.com/showhn.html. I've replaced "Show HN" with "Tell HN" above.
Also, if you're getting these stories from HN then I would ask you to link to the HN submissions.
We totally support the goal of bringing attention to great stories that haven't gotten significant attention yet. Other people have worked on this problem (e.g. https://news.ycombinator.com/item?id=23392049), and in fact we spend a good deal of time doing the same thing (https://news.ycombinator.com/item?id=26998308). But it's not good for community to sever the links to the source where you found them. In fact, at first glance it felt exploitative to me—a bit like kicking the ladder out from under you—but on reflection I'm sure that's not what you intended.
If you include the HN sources the way that polote has done here: https://hnblogs.substack.com/p/hn-blogs-24121, that would suffice.
p.s. It looks great and I'm going to subscribe!
Btw, it's been on our list for a good 7 years to ask people to confirm that they've read the rules before posting a Show HN, but I always hold back because it would break a certain precedent. HN doesn't tend to use nag screens and so on.
Exploitative? Of who? The submitters?
I only ask because as both a submitter and content creator I have zero problem with anyone finding items through /newest and sharing them wherever they like, credit back to HN or not, and am unsure who benefits from dissuading this.
Oh, this is an interesting qualifier. The newsletter is not based solely on HN. There are other sources as well. How much is from HN? I'm not actually sure myself, but I'm gonna attribute some HN links (where applicable) and we'll see.
Whatever sources you're systematically drawing on, each deserves acknowledging. By "systematically" I just mean that when people randomly pass around one-off links to random articles, there's no need to be particularly scrupulous about provenance, but when they're building projects on other platforms, that's different, and then there is such a need. Otherwise the balance between giving and taking is out of whack.
The "taking", in this case, is using HN (and other aggregators) as a source of links to interesting stories; this needs to be balanced by some "giving", and on the web, the well-established way to do that is to link back to the place you found it. That way, good things can flow in both directions: your readers get interesting things to read, but they also get to know about a community they might like, and the community has a chance to gain new readers who might enjoy it.
If you don't do that, then the claim you're implicitly making is that you scoured the entire web looking for those articles, rather than drawing them from other aggregators that already did that. That woudl be false and misleading—it's "taking" too much, in the above sense. What you're actually doing, which is looking at existing aggregators to find good content that got overlooked there, is (a) genuine work and (b) a wonderful contribution—and that's the work and contribution that you deserve and should take credit for. I don't mean to criticize that or minimize it! The existence of many good articles that get overlooked shows how much it's needed. It's simply that each contribution needs its proper acknowledgment.