Bad take. I opened the Github issue linked. For us it represented, at times, thousands of requests per second across multiple users. And that was with affected users getting IP-banned temporarily. Some of which were…
It's not even 100Mbps sustained. That is nowhere near 30k/year or you're getting ripped off. For 3k/month you can get a good quality 10Gbps link. That's 3.2PB with a P.
Well, lucky you. Or unlucky me and everyone I know running a large website. Guess we’ll never know.
I’m aware. Doesn’t make it sting less being in the receiving end of attacks all the time and seeing everyone collectively shrug.
> Trust me when I say that you don't want the ISP's to inspect web traffic. They do already. DPI on port 53 for DNS blocks or SNI inspection are common place. So are IP blocks. > If you want traffic, you need to be…
> Traditionally, a botnet can be compromised (at least largely) of actual consumer devices unknowingly making requests on their owners' behalf. And I do count that in. Just because a user is the source of an attack…
IP reputation is already a thing. And plenty enough ASNs are well-known for willfully hosting C2 servers and spam, DoS, etc sources…
These blogposts document the attack. Documenting it and acting in it are different. There’s no practical action being taken besides « use our profucts cause we can tank it for you » here. The mitigations listed are…
> What happens when you log an attack from a device that is attacking you from a school or business WiFi network? Block the whole IP forever? No, but for a day perhaps. > What if the user is on a CGNAT. Are you going to…
> just with everything production-grade, the average enterprise just isn't ready to deal with all the upfront cost to run your entire computing solution That’s not a fair point. We’re not even trying to make the…
You’re completely wrong. All large sites regularly get attacked. The average skiddie’s motivations are that they’re bored. So they DoS a site they use regularly just to see. Heck they generally don’t even mean to cause…
Thing is, don’t care. The problem is that ISPs whose customers are originating the attacks from don’t give a shit. If we have to give up 1% of legitimate traffic to thwart 90% of attacks, it is a good deal. If you and…
You can’t. If your webserver receives 400m rps it dies, end if story. Mitigations are just that, mitigations. They are as effective as buying a better door lock to protect your apartment from a nuke.
> k8s cluster exists solely to handle their monitoring and logging Does image processing, runs our analytics, runs our Sentry, runs our gitlab-ci runners, and quite a few other things not mentioned expressly > which…
As I said, it's not so much that we ask that data to be fetched -- it is there in the first place, and pulled from Elasticsearch, not a SQL database Because of this model, we also make sure that Elasticsearch merely…
Definitely needs optimising for user experience indeed! However the serving of this JS has nearly no cost to us (as they are cached at the edge by DDoS-Guard and the frontend is otherwise entirely static on our end)
That's very close to how MD@H works, but it also has a time component and tokens are not generated by our main backends, so it'd require a separate internal http call per chapter
> The real problem is that generating that much JSON is very "heavy" on servers. Lots and lots of small object allocations, which gives the garbage collector a ton of work to do. It's also expensive to decode on the…
> Would definitely like to hear more about their dev environment, how it is different from prod, and how they handle the differences. It's honestly quite boringly similar (hence why it's only vaguely alluded to in the…
Not correct, we generate 2 thumbnails sizes for every cover -- if the site loads full-size anywhere by default (rather than when you expand it) it's definitely a bug!
> This is great in terms of success as a website, but it's underwhelming in terms of describing a technical problem. A bit of an intro punchline, even though I agree it admittedly doesn't say much on itself :) Fwiw most…
Hi, we're trying to lower the requests:pageview ratio in general, but for what it's worth this article essentially: - ignores the vast majority of "image serving" (most is handled by DDG and our custom CDN) - the JS…
Bad take. I opened the Github issue linked. For us it represented, at times, thousands of requests per second across multiple users. And that was with affected users getting IP-banned temporarily. Some of which were…
It's not even 100Mbps sustained. That is nowhere near 30k/year or you're getting ripped off. For 3k/month you can get a good quality 10Gbps link. That's 3.2PB with a P.
Well, lucky you. Or unlucky me and everyone I know running a large website. Guess we’ll never know.
I’m aware. Doesn’t make it sting less being in the receiving end of attacks all the time and seeing everyone collectively shrug.
> Trust me when I say that you don't want the ISP's to inspect web traffic. They do already. DPI on port 53 for DNS blocks or SNI inspection are common place. So are IP blocks. > If you want traffic, you need to be…
> Traditionally, a botnet can be compromised (at least largely) of actual consumer devices unknowingly making requests on their owners' behalf. And I do count that in. Just because a user is the source of an attack…
IP reputation is already a thing. And plenty enough ASNs are well-known for willfully hosting C2 servers and spam, DoS, etc sources…
These blogposts document the attack. Documenting it and acting in it are different. There’s no practical action being taken besides « use our profucts cause we can tank it for you » here. The mitigations listed are…
> What happens when you log an attack from a device that is attacking you from a school or business WiFi network? Block the whole IP forever? No, but for a day perhaps. > What if the user is on a CGNAT. Are you going to…
> just with everything production-grade, the average enterprise just isn't ready to deal with all the upfront cost to run your entire computing solution That’s not a fair point. We’re not even trying to make the…
You’re completely wrong. All large sites regularly get attacked. The average skiddie’s motivations are that they’re bored. So they DoS a site they use regularly just to see. Heck they generally don’t even mean to cause…
Thing is, don’t care. The problem is that ISPs whose customers are originating the attacks from don’t give a shit. If we have to give up 1% of legitimate traffic to thwart 90% of attacks, it is a good deal. If you and…
You can’t. If your webserver receives 400m rps it dies, end if story. Mitigations are just that, mitigations. They are as effective as buying a better door lock to protect your apartment from a nuke.
> k8s cluster exists solely to handle their monitoring and logging Does image processing, runs our analytics, runs our Sentry, runs our gitlab-ci runners, and quite a few other things not mentioned expressly > which…
As I said, it's not so much that we ask that data to be fetched -- it is there in the first place, and pulled from Elasticsearch, not a SQL database Because of this model, we also make sure that Elasticsearch merely…
Definitely needs optimising for user experience indeed! However the serving of this JS has nearly no cost to us (as they are cached at the edge by DDoS-Guard and the frontend is otherwise entirely static on our end)
That's very close to how MD@H works, but it also has a time component and tokens are not generated by our main backends, so it'd require a separate internal http call per chapter
> The real problem is that generating that much JSON is very "heavy" on servers. Lots and lots of small object allocations, which gives the garbage collector a ton of work to do. It's also expensive to decode on the…
> Would definitely like to hear more about their dev environment, how it is different from prod, and how they handle the differences. It's honestly quite boringly similar (hence why it's only vaguely alluded to in the…
Not correct, we generate 2 thumbnails sizes for every cover -- if the site loads full-size anywhere by default (rather than when you expand it) it's definitely a bug!
> This is great in terms of success as a website, but it's underwhelming in terms of describing a technical problem. A bit of an intro punchline, even though I agree it admittedly doesn't say much on itself :) Fwiw most…
Hi, we're trying to lower the requests:pageview ratio in general, but for what it's worth this article essentially: - ignores the vast majority of "image serving" (most is handled by DDG and our custom CDN) - the JS…