Ask HN: Consider banning paywalled articles like those from wsj.com please

121 points by NicoJuicy ↗ HN
I can't read the paywalled articles on wsj.com anymore. So it's breaking my web experience of reading HN.

The first post on the page is mostly a TLDR or a link to an alternative article instead of a real comment. For example the current #1 post : Samsung to Permanently Discontinue Galaxy Note 7 Smartphone (wsj.com) - https://gyazo.com/440858bfcc2f94d912a087ca2ef7ce26

There will always be a discussion about paywalled articles and or alternative sources, which takes the discussion away from the topic and there is a high chance that this will be a top comment ( because of upvotes).

The current #1 link and #7 link are currently unusable to me : https://gyazo.com/578b105ce67f9b6d14cefe0dfeb43efb

The trick with bypassing it through the web doesn't work for me anymore ( and multiple others). This was good in the past, but not anymore.

An alternative like automaticly hiding articles from wsj.com would be nice, but i think it's more effectively to just ban the domain? Because it has no use for many HN-readers.

Suggested alternative: Add (paywall) automaticly to the link

Please discuss or give your opinion?

65 comments

[ 2.2 ms ] story [ 139 ms ] thread
Slap (paywall) into the article title. Then people who find it a problem know to skip it.
The trouble is, that's locking a set of people out of the debate entirely, which is surely not what we want to encourage. Much better to have an alternative, freely-accessible article from a different source.
It already locks people out of the debate :)
This.

I'm one of the people affected; I am already shut out of the discussion, and when I am not on a fast unmetered broadband connection, the wait and the advertising thrown at me instead of the content that was promised feels like I am being punished for following HN links, every time.

I'm not suggesting it is a perfect solution, but a heads-up would be strictly better than what we have now.

(I don't much like the idea of copying paywalled content elsewhere, as suggested by other commenters - there are obvious legality issues, and also it seems morally dubious at best; the content provider has made the decision that they don't want non-paying customers to read their stuff, and however we feel about this, we ought to respect it - just as my decision not to pay for some kinds of content (and hence not receive that content) should be respected as a valid choice instead of being punished.)

Why is it so much better to encourage blogspam? Which is what you'll get when the topic is of worthwhile substance, such that there is no real alternative besides an aggregator?

Why shouldn't the burden be on the reader to jump into incognito (i.e. non-plugin) mode to read an article? Virtually everyone on HN has the technical chops to switch between an incognito browser instance and whatever they're using to read HN.

Yeah, I've made a point about incognito mode in a different comment - I agree, that's probably the best workaround we have right now. I wouldn't encourage 'blogspam', but I'd be surprised if 99% of paywalled articles weren't also covered to more-or-less the same quality elsewhere. The topic that kickstarted this discussion - Samsung's recall of the Note 7 - has been covered widely; theguardian.com is always a reasonable source of general-interest tech stories.
I block marketing domains in my daily-updated /etc/hosts file. This is a much more performant and much safer way to block malware and adware than using a browser plugin. Yet, your suggestion to just shut off your plugin helps me none.
Er, what help do you need? If you aren't using a plugin, then visiting the paywall article via Google should work for you as etc/hosts does not affect the referer attribute
I would prefer this so that people who do want to spend time looking for changed referrers or cached copies and such can do so, while others can just skip it or see if they can submit a non-paywalled link. Let the submissions float up the ranks based on how people react to the nuisances of dealing with such sites.
I agree with the proposal to ban paywalled sites. Workarounds are annoying and/or temporary. HN should be about open discussion of freely available content; we shouldn't be encouraging the ghetto-ification of the web. The typical paywalled sites are almost never the only (or even the best) source of an article.
So having this mentality, HN should also ban any links to non-free books ( e.g. amazon ), links to apps and any other websites that offer a paid service / content, which you have to consume in order to be an active participant in the conversation, happening in the comments.

I don't agree with this position.

TBH, I don't see how a link to a book page on amazon would result in a productive discussion on HN - is there a precedent? I wouldn't ban apps/websites that offer a paid service, only those that require one. And I'm willing to row back slightly from an outright ban, as I've said in other comments.
Everyone sees the same there, no problem with that.

If a blog has paid content that is only available for some people, that is similar to this discussion. Not your example in my opinion

I would much prefer HN to link to quality sites, especially if they use few ads.
I agree, most WSJ articles are available on other news pages and the web workaround doesn't work for me either (since a few days).
I support this proposal. I support any independent company's right to perform their business how they want. I also support that popular sites that encourage open dialog and discussion should not include companies that disallow public viewing of their content.

Let's also extend this to companies that prevent those with content blockers from viewing their pages. Forbes is one, and techcrunch is another. Techcrunch doesn't block content, but their site UI is messed up enough that when content blockers are used that it makes their site unusable.

Techcrunch looks great in Reader mode.
Does the web link work from an incognito session? I suspect that WSJ just sets a cookie on first visit to show the paywall on subsequent visits.

I personally agree with the current HN paywall policy. If the Google workaround doesn't work, then WSJ will get banned from Google, which is a far harsher penalty than getting banned from HN.

It's probably the best workaround that we have right now. The trouble is, it's an additional cognitive load to determine for each and every link whether it's one that requires incognito mode. If there's a good browser plugin that supports automatically opening links from listed domains in a new incognito window, that would probably be better than the current 'refer via google' workaround.
Whoever can read them, take a screenshot and upload it to BayImg. Solves it.
That might help to an extent, but it really doesn't solve the problem properly - an image of some text is a far inferior reading experience to the actual text itself.
(comment deleted)
I disagree with banning sites that have leaky paywalls. Sites that do good journalism deserve the pageviews. We shouldn't incentivize blogspam just because it's convenient. In the case of the WSJ, they were the outlet that did the investigative work that broke Theranos, among many other stories.

The question boils down to whether the paywall is leaky enough. The "web" option, which helps the user find the article via Google because the WSJ paywall let's through the Google-referer, still works for me and many others, but apparently, not for other users. That's a strange technical quirk. I've tried emulating it on desktop vs. mobile, and using a VPN for different parts of the world, and the Google-referer-workaround still works fine for me.

So why can't some users get it to work? Could they be using a plugin that blocks the referer? Is the use of such a plugin enough of a common-case to block good journalism for the rest of the users on HN?

An alternative is to plug the url into archive.is, which I've found to always work for me.
About 6 months ago, I reproduced this problem, cleared my cookies, and then it worked. Although I've never been able to reproduce the problem after that.
I'd like to see linking to paywalled sites discouraged, but banning them might be too inflexible. Perhaps a change to the submission form that suggests that the submitter looks for alternative sources first, and appends "(paywalled)" to the title if they opt to proceed.

Paywalled mass-media articles (e.g. WSJ) are very rarely worth submitting. Articles in paywalled journals (e.g. Nature) where the abstract is accessible may well be worth it. But it comes down to judgement.

Finally, I agree that (for the type of topics that HN concerns itself with) free access to information should be encouraged.

I like this suggestion as a pragmatic compromise. There could even be consideration of weighting paywalled articles slightly lower in the general hn algorithm.
I agree.

The BBC do a similar thing with their football news roundup from the daily papers with "<short overview> <paper name> - subscription required"

This doesn't stop people who do subscribe to the site from reading the content linked to and gives a heads up for people who don't.

The Samsung story isn't exclusive to the WSJ and there are plenty of other free to access news sites that would have been better for the submission. If we ban paywalled sites entirely it would prevent submissions of exclusive content (which could be an interview or journal)

I agree with this proposal. Appending "(paywalled)" to the title at least lets me know this info exists and, if it's something I'm likely to be interested in, I can go to google or some other web source.
Maybe a [paywall] tag in the title could help.
I agree with this. I'm the submitter of one the OP's example links from yesterday. I was aware of the impact the paywall would have, which is why I tried to helpfully summarize in this comment:

https://news.ycombinator.com/item?id=12680419

The mods allowed that to be the top comment in the thread for a while to help loop in people who couldn't otherwise read the content.

I'd like to also emphasize that sometimes paywalled sources are the only sources. In this specific case, WSJ was the first source to break the news on the Theranos lawsuit story I posted. I subscribe to WSJ and I received the notification when the article went up, but I looked for other sources before posting. It's not uncommon for WSJ to be the first or only source for new articles.

I would be in favor of adding "Paywall" to these links as well, but I also think that banning paywalled sources is too inflexible.

If it is a paywalled article in a journal, just submit the sci-hub link.
No, it's a bad idea. Pay for good content.
I don't know if "banning" is quite right, sometimes the article only exists behind such a structure, but may be worthwhile for the community to discuss. Mods already replace higher quality links after they show up in the discussion. The same could happen by submitting a prominent paywalled source to get the discussion rolling, and then as a better source is made or discovered switch over to it.

And the comments on HN seem to be more coupled to an event than any given particular article, meta-metadiscussion aside.

I don't think these sights should be banned. WSJ and NYT in particular produce excellent journalism and the digital subscriptions are actually quite cheap, even if you're on a budget. If anything we should ban tabloids like Business Insider which mostly produce summaries of other sights and listicles.
For WSJ: Open the web link in an incognito window; click the search result. Does that not work?
I propose that paywalled article be allowed, but as soon as a comment recommends a non-paywalled article for the same story, we switch to that.

HN already replaces article URLs with better ones, so the above will work through the old process with an additional parameter for what's considered "better" for HN.

I like the idea in theory, but in practise it always takes some time till the link gets replaced and it's manual labor for HN.

I don't think it's something i as a developer would want to see as a solution ( just my 2 cents)

I'd like to see more summaries on HN. I'm often reading HN on my phone, and the connection is often too slow to read the actual article.

A summary will also help with paywalled articles.

I think a submitter of paywalled content should make sure a summary/excerpt exists on the site linked, or they should include one themselves. Sounds like a fair compromise between not being able to link and not being able to read, while still stimulating some discussion.
And ban quora while you're at it? Ah, wishful thinking of course.

I do agree that inaccessible sites are of little value for discussions and would gladly have a visual indicator of these sites to avoid the click, close, flag response.

Protip: setting your user agent to GoogleBot eliminates a lot of bullshit like this. Most sites, including Quora, won't hold out on GoogleBot, although the WSJ does. I guess they don't care if 90% of their article doesn't get indexed, that's pretty brave.

There is a plugin for Chrome that makes that easy: https://chrome.google.com/webstore/detail/user-agent-switche...

I don't think they want us to have this discussion :) - http://imgur.com/a/GdMkz

According to me, this should be a top discussion. But the ranking algorithm seems to be way off, like we have been punished in the algorithm.

Any thoughts what this could be?

Ask HN posts probably have a higher bar as they're often lower quality or less interesting than news?
I thought the "older" published algorithm was only an algorithm with sum of upvotes on the topic (and comments) with a time decay.

Seems reasonable, but i don't think it true ( considering Ask HN: are just texts in the front of the title).

Edit: It's definatly not in the algorithm, it's a manual ban of some sort :)

Is HN open source or something? How is it that everyone knows this algorithm?
news.arc is open source, however, HN's voting ring and antispam features are not.
This post was subjected to several ranking penalties, not least of which were user flags and the flamewar software. Many meta-discussions end up this way because they were off-topic for HN by virtue of not being intellectually interesting, which is why we're here.

The paywall issue in particular has received plenty of discussion already: https://news.ycombinator.com/item?id=10178989.

Never ever saw that article, thanks
Banning seems extreme. We have mechanisms on HN to self-moderate -- If you do not like paywalled articles, do not upvote them or post them. But as long as other people are posting and upvoting, it means some part of this community feels they have value.

So this discussion is great - lets make sure everyone takes the time to give it thought. But after those thoughts have had time to percolate, the actions of the community will self-correct. Or not.

How about a host filter in the user options? At least that way people with a subscription can see the link.
This is one of the oldest arguments on Hacker News.

https://news.ycombinator.com/item?id=10178989

Note that "incognito window" is one of the "standard workarounds" 'dang mentions in that thread.

That works for other paywalls but not WSJ (or at least, not consistently: sometimes it works, but it usually doesn't).
It works reliably for me. Incognito, click the "web" link on the HN byline to generate the Google search. WSJ checks for cookies and referer.
> I can't read the paywalled articles on wsj.com anymore. So it's breaking my web experience of reading HN.

If a group of people at a party were talking about a book you had not read or a movie you had not seen, would that be breaking your party experience?

Of course not. You'd go converse with a different group that was talking about some other book or movie, or talking about sports or something.

You can do the same thing here. You have 30 submissions on the front page alone to choose from, and even more if you can bring yourself to hit the "more" link at the bottom of the page.

And unlike the book/movie discussion at a party, if you do go into the comments on an article you have not read you are unlikely to get any spoilers that will ruin it for you if you do later find the article somewhere you can read.

> An alternative like automaticly hiding articles from wsj.com would be nice, but i think it's more effectively to just ban the domain? Because it has no use for many HN-readers.

Every highly technical article discussed on HN is of no use to many HN readers. Do you think we should ban those, too?

Also, the utility of HN submissions is usually in the comments, not the article itself. The comments are usually quite useful even without reading the article. The article is merely a launching point.

> Also, the utility of HN submissions is usually in the comments, not the article itself.

Well in order to properly comment and have a meaningful discussion the link should be read at first place.

Your analogy is broken, what people are doing at your party is discussing a movie they've only seen the trailer of.

Your comparison is wrong. It's like some people at a party have a key to a door and i and many others haven't. So yes, it's a "party breaker". Not being able to read the top articles (#1 and #7 )is also a party breaker. What would happen if all the articles where behind a paywall.

If a technical article is here, i choose if it's interesting to me and upvote it accordingly. If i can't access content, then i have not choosen. The website has chosen for me and i don't think it fits in the spirit of HN.

I have not said to just ban the domain. I have said to consider it and my last sentence was to discuss it. I even added an alternative ( like add paywall to it), because it's clearly a problem for a lot of people.

Please don't change my words

> If a group of people at a party were talking about a book > you had not read or a movie you had not seen, would that be > breaking your party experience?

Not really, but, while snappy, that's not really a good depiction of what's happening here.

A better analogy might be groups of people at a large exhibition discussing what's happening in the various booths. You're welcomed to go look in the relevant booth then join the discussion. Most of the booths you can just wander up to and take a look inside, but sometimes you walk over and poke your head in and the booth bouncer looks at your visitor badge, slaps you in the face to attract your attention and demands you fill out a bunch of forms and pay them before they'll let you in.

The wasted walk and the slap in the face kinda ruin your experience, so you ask if people discussing the slapping sort of booths might maybe warn people first, but they suggest you just put up with the slaps and move on if you don't want to pay up front for everything.

How does that make you feel?

If nobody can read the content of a link then no discussion can take place on HN therefore it shouldn't be on HN. I say ban paywalled articles and stop generating free traffic for these websites that don't want to share information at first place.
While we talk about how free apps are killing the market and making life difficult for an indie developer, we want to make it difficult for WSJ reporters too? If the story is broken or first reported by WSJ, example the suspicious cases one, by all means they should be the ones linked. At least that way people are aware of the good work WSJ is doing and maybe pay for the news.
The current policy is that articles from sites with easily-bypassable paywalls are allowed.

The WSJ paywall is not easily bypassable. The usual techniques to bypass paywalls are inconsistent with WSJ. Sometimes opening the 'web' link in an incognito window works, sometimes it doesn't. When it doesn't work, there's no way to get to the article.

The WSJ paywall should not be treated like the NYT paywall. It's a different animal, and it shouldn't be allowed here because it fails the easily-bypassable test.

Another solution is to search for the words in the headline. This works for me even in a non-incognito window. Very little effort required.