Tell HN: Stack Exchange (Stack Overflow) is now blocking Tor
I tried visiting Stack Exchange today using Tor Browser, and apparently my IP is blocked. I tried with several different circuits, and they're all blocked. Several people on IRC confirmed this as well.
I tried to contact them by e-mail and they didn't bother to respond, as is par for the course these days.
The full error message:
---
Access Denied
This IP address (185.220.101.37) has been blocked from access to our services. If you believe this to be in error, please contact us at team@stackexchange.com.
When contacting us, please include the following information in the email:
Method: block
XID: 820408900-HHN
IP: 185.220.101.37
X-Forwarded-For: 185.220.101.37
User-Agent: Mozilla/5.0 _Windows NT 10.0; rv:91.0_ Gecko/20100101 Firefox/91.0
Reason: Blocked.
Time: Tue, 08 Feb 2022 08:11:50 GMT
URL: tor.stackexchange.com/
Browser Location: https://tor.stackexchange.com/
217 comments
[ 3.2 ms ] story [ 281 ms ] threadThere have been DDoS attacks recently on Stack Overflow and Stack Exchange, I think it's quite plausible those have something to do with this. They still seem to be under active attack intermittently.
Stackoverflow pages have many dynamic components like vote counts, reputation points, sidebar related-questions-links, new comments, etc. An excerpt from their blog explains they can't cache the output : https://nickcraver.com/blog/2019/08/06/stack-overflow-how-we...
Even though Stackoverflow's website io access pattern has higher reads than writes, the resultant generated html is still not as static as Wikipedia pages. Even if they're efficiently using cpu to generate the dynamic elements, the cost of egress traffic may also be a factor.
All that said, I don't have any insight into what heuristics they use to block certain ip addresses.
The egress traffic is also trivial - a page seems to clock in at below 50k. If they're paying $0.02/GB, that's $1 / million requests.
More importantly, if it really were a DoS attack, there's way less obtrusive methods, such as a CAPTCHA or similar verification screen.
> variants of cache... anonymous, or not? mobile, or not? deflate, gzip, or no compression?
I don't understand why the markup would vary between mobile/desktop, or why they would even consider compression a varying factor to account for in cache. Maybe that's because their backends produce such specific variants that they have a hard time caching in the first place?
If 80% of pages are only requested every two weeks, then it doesn't make sense to cache those. There's still probably lots of stuff you can cache, such user profile/stats and question/answer scores for several minutes. You can also cache many parts of the markup that are not going to change often. I mean it's always even better when you cache the entire markup, but there can still be lots to gain if you cache only small bits that are expensive to acquire/template.
> But the cost of memory to store those strings (most large enough to go directly on the large object heap) is very non-trivial. And the cost of the garbage collector cleaning them up is also non-trivial.
It looks like their cache implementation could (should?) have been based on better foundations. First, i think the cache doesn't have to reside in memory: disk accesses are fast, certainly much faster than running a database query over the network which will need to access several files and cross-reference data with extra latency on top. Then, because if you're gonna store long-lived stuff in memory you should probably use a garbage collector based that's tailored for this usecase, not your language's (.Net) default GC... maybe Redis? Don't get me wrong, i find it pretty cool if some engineers want to develop a homebrew cache, but that sounds like a huge project in itself.
> the cost of egress traffic
I'm not aware of SO tech stack, but i'd be surprised if they have much egress fees. They're a very big site so they probably run off unmetered dedicated servers if not their own hardware on cheap transit. Who knows, they may even have their own AS and peer with other providers in some locations? From a quick request, it looks like stackoverflow.com is served from Fastly AS but i personally don't understand why: i don't remember seeing any heavy content (video/images) on SO so in that kind of situation a CDN would hurt more than help on slow links. Maybe that's because they bundle megabytes of javascript crap? Now that they're blocking tor users i can't check for myself :-)
Things like this https://news.ycombinator.com/item?id=26072025
Fixes, updating the app, blocking the image, blocking all requests with empty user agents.
------
1 person abused enough SO, they dig, find the culprit being someone using Tor network.
Fixes, identify the user and ask them to stop, block all traffic from Tor.
-------
Do you propose an alternative fix?
The new blocking completely blocks access to the site, not just posting. See for yourself; https://www.torproject.org/
Tor is next to useless for DDoS attacks, as it doesn't offer any amplification. For every byte you send in via TCP, you get one byte out. For attackers with large botnets, it doesn't make sense to DDOS over Tor, as the number of exiting IP addresses are limited and it's easy to block them all. It makes more sense to use your thousands of available botnet IP's that aren't on any lists.
In order to carry out these attacks, the attacker already controls enough machines to DDoS one of the largest websites on the internet. So, huge botnet or nation state. Now, we have one of the largest websites on the internet telling it's tech audience:
https://meta.stackexchange.com/questions/376060/update-on-th...They are normalizing the "workaround" of using an insecure IP address when Tor is inaccessible. This will lead to all non-secretive and non-illicit Tor usage to go back to the open insecure internet. Thus, everyone still using Tor "has something to hide" (as if that wasn't the case already). By forcing all but the most desperate users off Tor, Tor can be discredited as a nefarious tool.
I think this is a reach - after all, if you wanted to discredit a tool for people in oppressive regimes, people who care about their privacy, and people doing illegal things, why start with the programming help site? (I know SE has other sites, but the biggest ones are for tech help)
...That said, I could see this being a way LE could try and unmask a very high-value darkweb programmer. Still a reach.
My guess is they'd target websites that are important to TOR users, and tor.stackexchange.com is one of them. I also don't think they started with SE. TOR IPs are continually filtered by Google, Cloudflare, and other automated firewalls.
I'd assume that Stack Exchange might also get lots of bot traffic from TOR.
The DNS checking service is rate limited and adds latency, but there is a bulk exit list available here; https://check.torproject.org/torbulkexitlist (please don't use 3rd party blocklists, as those often contain middles and guards, even though traffic cannot leave those kind of relays).
But, before you block Tor, please consider that doing so will most likely block a number of legitimate users who you haven't noticed before - just to stop one jerk. Instead, you could restrict access to certain parts of your site for Tor visitors for example.
Bot traffic from Tor is usually of the easily detectable variety (as the attacker didn't have enough skill to build/acquire a botnet and get "clean" IP's),
Is their site really so sensitive as to make reading with Tor impossible?
Very much so, strange if people here don't understand that, even at the best of times APT's could be discovered down the track by their SO queries, now compare today, with heightened tensions and certain nuclear armed superpowers talking about going to war with each other.
How is this even slightly surprising? SO is vital shit, if you don't agree feel free to null route the site the next time you have a major incident at work :)
I still do not understand how blocking Tor helps here. People who are concerned about their security will either use mirror sites, or use data dumps such as what is available at archive.org, or simply not use the SO/SE content at all. The number of users who will abandon Tor and the protection it provides for the express purpose of visiting SO/SE is negligible.
This move will not increase the number of persons who see SO/SE adverts or who are trackable by SO/SE. It will also not decrease the number of persons who will be able to access SO/SE content. So I continue to be mystified about the rationale behind this policy change.
If SE executives are really concerned about spam and vandalism by anonymous actors, then SE could Tor users to post assets in escrow (e.g. Monero) before posting. Similarly, if SE executives are concerned about denial-of-service attacks, then SE could rate-limit the sites that are causing the attacks; Tor is not efficient for that kind of attack anyway. There is no sound argument that blocking Tor entirely would further the interests of SE users.
This is the act of a monopolist in secular decline.
If SE executives are really concerned about spam and vandalism by anonymous actors, then SE could Tor users to post assets in escrow (e.g. Monero) before posting. Similarly, if SE executives are concerned about denial-of-service attacks, then SE could rate-limit the sites that are causing the attacks; Tor is not efficient for that kind of attack anyway. There is no sound argument that blocking Tor entirely would further the interests of SE users.
A site looking to grow its influence would be more concerned with attracting new users than repelling them. This is the act of a monopolist in secular decline.
To your other point, how can one establish a 'reputable account' if it is not even possible to access the site in the first instance.
I don’t understand why Tor users would be interested in reputation at all, given the implicit identification it requires.
If you access any website through Tor (or proxies) you're already more suspicious than the average user. If enough people cause trouble through Tor exit node IPs, it's only natural they get blocked.
Akamai published an analysis that affirms this:
https://web.archive.org/web/20170317110115/https://www.akama...
The relevant part:
> "we concluded that approximately 1 in 380 http requests coming out of Tor is verified to be malicious, while only 1 in 11,500 http requests coming out of a non-Tor ip were verified to be malicious. In essence, an http request from a Tor ip is 30 times more likely to be a malicious attack than one that comes from a non-Tor ip."
For a serious site, the cost of allowing passive reading is going to be ~0.
“Legit” Users likely block ads, are unlikely to enter their credit card numbers because of MITM shenanigans and it’s one of the few browsers that takes non-fingerprintability of its users seriously.
We occasionally had people upload child porn, they did it over the public internet and not tor, our lead counsel was a former US district attorney, his hobby was doxing the uploaders and providing all the evidence and information to the authorities in a "ready to prosecute" package. I forget the exact number but I think he got almost a dozen people prosecuted and jailed.
Come up with a way for siteops to block someone and all their sockpuppet accounts, without knowing the underlying identity, and you’ll become a billionaire.
Without that, the only option we have today is deanonymization, which is a terrible option. We ought to do better.
But not allowing Tor users to not even read the website? What's the justification for that? You couldn't even perform DDOS over Tor as the network speed is too slow, so Tor activity can't even be a blimp in terms of usage activity for logged out users, so what's the deal here?
Given you only contacted them today presumably, I'd allow them at least a few days to respond...
That said, if they respond, I'll edit the post.
Also, dude, it was the end of the work done or close to midnight in most of the western world when you made this post.
[1]: https://stackoverflow.com/legal/trademark-guidance
I mean anyone who'd like to crawl/scrap a website can do it from home, with the help of friends, or by renting "resential VPN" services for a few bucks a month. Blocking Tor does not prevent scrapping.
Google should definitely resolve this problem on their side, but language models got huge and complex. It's hard to tell (especially with their scale) if a website if composed from paraphrased content of 20 other websites.
I'm kinda shocked how successful those "doorway pages" are at reaching the top and how long they live there.
This at least makes sense, why run the JavaScript in your scraper? It's easy to identify. What do your webserver access logs say?
I don't know how much Ye Olde PageRank still factors into Google results, but I'm assuming it's still a significant factor.
Not to be too cynical about it, but who serves the ads on those sites?
If it's Google Ads, then I think you have your answer as to why they are higher ranked.
https://github.com/quenhus/uBlock-Origin-dev-filter
There are 322 SO copycats alone.
https://www.brentozar.com/archive/2015/10/how-to-download-th...
Though that is a bit conspiracy-theory-y and I expect blocking Tor is more an issue of a small but very active number of people using it to create accounts to post anonymous spam or abuse. For that, which can be a very real problem, I would suggest a more fine-grained block: stop people from those addresses creating accounts, or posting from relatively new (or recently inactive) accounts, or perhaps somewhere in the middle: prevent connection from Tor addresses posting at all, but still let people use the site. That way you still block the abuse, but have less impact on others accessing via Tor.
StackExchange is composed of contributions from the Internet at large. I've appreciated over the years how StackExchange took this stewardship very seriously without hoarding it behind paywalls - the ethical thing to do.
One of the reasons I was motivated to contribute is because 10-12 years ago, putting tech questions in Google often led to Experts Exchange, a site that takes contributions from the public and makes you pay to see them. (Really need a StackExchange-type site to take on Pinterest now ...)
So ... no, they should not stop publishing updates. People need to bookmark sites and go directly to them instead of relying on an increasingly broken search engine that's past its heyday.
Well, if we are whopping them out on the table... 75.1K over the three I've used most, and a few thousand over a couple of others. Been around since the start, even got sent free shirts & bits in the early days for being in the top [what-ever-the-cutoff-was] on SF, SO, & DBA.SE.
> So ... no, they should not stop publishing updates.
I wouldn't mind if they stopped providing those dumps, I was just passing on that others suggest that they could (and by rights, they could, but I doubt they will, as my post (I though) said).
As long as the data stays available (just on the main sites is fine by me) under a CC BY-SA I'll keep contributing. They can't revoke the license for existing content, if they change it for future content (for the avoidance of doubt: I have no reason to believe they have any plans to) or the sites otherwise devolve (like others have in the past with rampant tracking and ads) I'll probably bugger off.
The "data" is under creative commons. It does not belong to them.
Example: years ago at Google we had proxies for Web traffic. Around lunch time you could hit an issue with HN where the traffic was blocked for too many visits from the same IP. There were thousands of engineers and they were concentrated on 1-2 proxy IPs.
A lot of systems tackle potential abuse (eg brute forcing passwords, DoS attacks) with a token and bucket algorithm. Example: a given source IP might get 20 tokens. Each visit uses a token and those tokens are regenerated 1 per 2 seconds.
So with a Tor exit node you may just be hitting abuse protection and that could happen on the CDN, gateway servers, internal servers and so on.
Things like Deletionpedia and Wikiwand provide an essential service by reducing the cost of switching and thus the power of hosting services like Wikimedia, but I don't know of anything comparable for Stack Exchange.
It was more than ten years ago but I was seriously spamming one of those reddit-like sites for software developers. I made about 1000 fake accounts with what looked like real names and photographs but if somebody looked close they might have noticed I made no effort to match the gender of the name to the gender of the photograph.
The accounts generated a good amount of "cover traffic" voting for articles that weren't mine but I would give one of my articles somewhere between 20 to 80 votes that would usually recruit an even larger number of real votes. Once in a while somebody would say something like "That's a pretty good blog but not that good, I can't see how he's possibly getting so many votes for every single post"
My take was that what I was doing was pretty safe because my fake users were actually pretty well behaved and there wasn't going to be any real perception that there was a problem. Even so I was using Tor to make sure that my requests were not coming from the same IP address and it wouldn't be so obvious what I was up to.
I lost the database with the fake accounts in a hard drive crash and I wasn't about to push the issue because I had also attracted some attention from the F.B.I. which knew I (or at least my alter ego) was making trouble with Tor so they tried to entrap me into distributing child porn over Tor. I told them no of course ("the FBI in my country sees that as an important enforcement priority") but that did make me lose interest in Tor.
Back then Wikipedia would require you to log in to make any edits over Tor because they didn't want to deal with that kind of BS either. It's not unusual for many kinds of sites to have restrictions on Tor users for the same reason.
My last covert web crawling project involved making a machine image that could be deployed in any AWS zone to do about 200 requests and then pop up in some other place. On one level it looks like you are getting hit from Japan and Singapore and Germany, and... But on another level it is all AWS.
At some point they looked at their logs and freaked out and made changes to make it fundamentally harder to crawl out their site. We've got the running gag today that it's not safe for me to go to Delaware now. But practically a lot of sites block AWS addresses for the same reason they block Tor. It's just way too easy for a third rate hacker wannabee to seem to be coming from too many IP address that way.
What if you had connected? What if an endpoint with a different IP can connect?
The standard here ought to be that the title accurately reflect the content. This post does not.
I don't take HN as the kind of place where people want to read speculation masquerading as fact.
I get they were acquired, but why would you want to shutdown a revenue generating business has been able to uniquely capture the attention of a highly skilled worker who's difficult to hire.
(FD: I am actually a worker of a games company owned by Tencent in Europe)
You have to be careful. Your friend could be Chinese. You never know.
https://meta.stackexchange.com/questions/376060/update-on-th...
For the past month Stack Overflow has been hit by weekly DDoS attacks that progressively grew in size and scope. In each incident, the attacker(s) have been changing their methodology and responding to our countermeasures. Initially we were able to detect and mitigate the attacks before any performance degradation could be noticed but the latest attacks ramped up very quickly and the site was brought down before we could react.
While we cannot go into specifics on each attack in order to maintain opsec and not tip off the attackers, we can say that each individual attack has been using different IP addresses and targeted different aspects of the site. During an outage, our top priority is always getting the site back up and running. After traffic has been stabilized, we perform a post mortem for the incident where we assess and improve upon the actions we have taken.
During the outage last Sunday, we noticed that a large amount of the DDoS traffic originated from Tor exit nodes. The decision to block Tor exit nodes did not come lightly, in fact Teresa, our CTO was on the call when we discussed remediation methods. Due to the persistent nature of the attack and our desire to bring the site back up as fast as possible we made the decision to block all DDoS traffic endpoints, including these Tor exit nodes.
We did not target, nor set out to block all traffic from Tor, that’s not something Stack has ever done. However, due to the shared nature of Tor exit nodes, some of them were also routing DDoS attacks to our sites and were blocked. We have tried removing these blocks between attacks but this action has resulted in further site outages as DDoS efforts continue to originate from these exit nodes. Unfortunately blocking the Tor exit nodes also blocks legitimate users from using them. An immediate solution for users who find themselves blocked is to access our site from other IP addresses, via home internet, work internet, or other VPN services.
We are continuing to evaluate the situation and will keep our community updated. Thank you for your patience and understanding.
Up through the mid-2010's, people posted amateur porn largely for personal fun and ego boost. Occasionally, regular people would try to make a buck. But it was mostly people posting selfies to /r/gonewild because they enjoyed the attention. Then OnlyFans came out, and now virtually anyone posting amateur adult material can be assumed to be driving traffic to a monetization channel.
Sorry for the tangents. It just strikes me how normalized "side hustle" culture has become. It was not very long ago at all that it wouldn't even occur to most of us to question a DDOS attacker's "motive".
Edit: maybe someone's scraper was too efficient.
There are dozens of sites like this. I even came across YouTube channels with thousands and thousands videos that created slideshows with elevator music that presented SO question and answers one sentence at a time. You can convert SO posts to blogs fairly easily and they rank.
I am going to guess that some people are trying to scrape the stack overflow data to make their own spinoff blog versions of it.
An interesting thought, but it doesn't fit with the description provided by SO of an evolving DDoS attack that originates from different networks in response to mitigations. It's not scraping. It's an attempt to deny service.
They are not scraping. The secondary economic exploitation is expected. The copy sites (attempt to) capture value that the primary is unable to.
Despite Google becoming worse overall, I noticed that the copy sites are consistently ranked lower than the original. What is your search engine experience? I have not encountered a robo Youtube video as you have described.
[1] https://www.kiwix.org/en/
[2] https://wiki.kiwix.org/wiki/Content_in_all_languages
--- App Privacy
The developer, Wikimedia CH, has not provided details about its privacy practices and handling of data to Apple. For more information, see the developer’s privacy policy.
----
... then you click the link for "developer's privacy policy" and it goes to a 404 at http://www.kiwix.org/impressum/
... then you go find the privacy policy on the website at https://www.kiwix.org/en/legal/privacy-policy/ and it says:
"This privacy policy applies to the website www.kiwix.org only. It does not cover our subsites like wiki.kiwix.org or download.kiwix.org."
If you can get the app and the database on tor, then you're golden.
Then the onion traffic can be handled differently (maybe DDoSing the .onion will only affect Tor users, or the resulting .onion resource usage is throttled). Worst case a readonly version of the site would be better than nothing.
Something similar could be done with the existing tor exit node blocklist, but maybe this is easier to separate from existing infrastructure.
[0] https://community.torproject.org/onion-services/advanced/oni...
If there was a good way, Cloudflare would have solved it. Cloudflare hasn’t been enough for my company. We had to add temporary Geo-IP blocks just to get the site up.
ie. I don't want Cloudflare CEO popping up here gloating about how they're fighting for the cause of tor users! F' that if they don't know it's there they can't take credit for it either. That's the tor I want.
Anyone for whom this is a solution has already tried it.
Tor is designed more as a proxy to the cleaner, but I2P is focused on internal services, this it would take extra steps to use it for DOS attacks against plain ok websites. If you want to be available to I2P users, you basically opt-in to the network, as opposed to Tor's effectively opt-out approach. I2Ps design is supposedly better at dealing with DDOS attacks, but I don't exactly know the specifics of how that is.
Overall, I think Tor will decline in usage over the next decade because it's too easy to use it nefariously and more institutions will find ways to block it.
You could detect and block Tor traffic by blocking the exit node IPs. They provide an API that lets you get all exit node IPs so that you could block them: https://check.torproject.org/torbulkexitlist
They are putting no effort in making this hard for site admins.