455 comments

[ 3.1 ms ] story [ 364 ms ] thread
I would like to see stats from Tier1/Tier2/IX for that. Krebs claims it's 665Gbit/s https://twitter.com/briankrebs/status/778404352285405188 Such attack must be visible in many places, however not a single major ISP reported that in mailing list. Previous smaller attacks were reported 'slowing down' some regional ISPs. Perhaps ISPs got better.
theoretically if it was a well distributed bot net then it could be a few megs from a TON of different sources.. and since akami had their own ASN it could just be jamming up Akami and not other AS
I've seen the graphs from prolexic, the claims are legit
In a world of 10 and 40 gigabit NICs, why is 665 considered big?
Because you have to classify and filter out the spam packets before they reach the intended host and content, which is really hard to do at line rate, especially if you also plan to serve useful traffic.
I really like this comment. I think you're saying that moving data is easy, while computing the data hasn't kept up?

Then again, I could be reading into this too much, and the computing part has always been a bottleneck at backbone level.

It seems to me that he's saying in order to move data at volume, you have to compute on it.

Computation is tricky. Per-bit, if you can handle the network input, you're probably able to fire packets up to the OS layer.

But when you need to run stats on the incoming data, e.g. an ML classifier of "bad/not bad" or "stop/passthrough", you might be O(n^2) or worse. Moore's can't hang.

Interesting. What ML algos are used to classify packets against DDoS?
I think that's the secret sauce for these quys. I'd be surprised if you can find out a lot about current techniques without signing an NDA and leaving your mobile phone in a box at security.
None, really. It's mostly filters against common types of attacks at L3/L4, then OODA. Variations from normal get looked at and custom filters applied as appropriate.

And of course, there's lots of NOC to NOC back channel comms around this stuff constantly to stay relatively on top of things.

Not sure if this applies to DDoS, but a baseline ML method for security is outlier detection. For example, (1) you get a dataset that is mostly "good" data, with some "bad" data (2) you cluster it using something fast like k-means (3) data points are labelled as outliers if they fall further than some threshold from a cluster center.
Linespeed is pretty fast, and there's a lot moving through. Routers are a bit like GNU grep -- they way to be fast is to not touch most of the data.

The more you have to touch, and the deeper you have to touch it, the more expensive it gets.

The real trick is figuring out what's good, what's evil, and downrating the latter whilst allowing the good. Given peering relations, BGP routing, and the sorry state of much of those protocols, tracing problems to their source, quickly, and getting a useful response, is difficult.

At line rate, with millions of small packets coming in every second, even counting the number of packets per flow, a prerequisite for some of the simplest mitigation strategies, is really hard (often requires either expensive hardware like ternary content-addressable memory, or exotic data structures like counter braids that are very expensive to decode). A lot of the time people try to get around this by probabilistic sampling, etc. Similarly, MICA's ability to handle key value lookups at line rate on commodity hardware was considered a big success in the database community: https://www.usenix.org/node/179748. Hopefully that should be a good indicator of the challenges inherent in performing any sort of nontrivial computation at line rate, even really, really simple computation.

(This is why most DDOS mitigation strategies involve getting peers to load balance their traffic when it's still manageable, rather than buying bigger and bigger pipes; it's also why ultimately a large part of the responsibility for handling DDOS attacks rests on the shoulders of ISPs).

I still don't get it. Why not just absorb it? Serving static HTML and related data is fast. A single rack in a datacenter could easily serve this with out breaking a sweat.

Sure, this costs Akamai money they don't want to spend, but is such an attack noteworthy? Eh.

You'd still be filling the pipe. And possibly filling many small(er) pipes at interconnection points. The last meters of pipes into a rack isn't where the issues are.
If the tiny "GET" requests are flooding in at 1Tbps, then the responses are going to be orders of magnitude larger, maybe somewhere around 500Tbps
I'm curious, what sort of datacenter racks do you run that can serve 665 Gbps of traffic? For instance, Google's ToR switches (in its own datacenters) as of Jupiter (2012) were 16x40G, which is just 640 Gbps. Obviously, all of Google can serve a lot more than that, and you can get absurd stuff like 640-port Infiniband etc., but you seem to have pretty unrealistic ideas about network capacity. And, as another comment pointed out, that's just the input for this attack... the output would be a lot larger.
you need 665 gigabits of idle capacity to... wherever the attacker is coming from. If the attacker can send five gigabits from some town in vermont, and your provider(s) can't get five gigabits plus normal traffic from that town in Vermont to your scrubbers, then legitimate users within those networks will be denied service, even if users on other networks are just fine.

Essentially, it comes down to the fact that getting packets from point a to point b requires a lot of cooperation, and cooperation is difficult. Yes, yes, if you bought me the fiber, I could build you a 665 gigabit network, on the kind of money that a nerd could come up with, (not counting the fiber) but interconnecting that network with other people's networks? yeah, that's gonna cost you. Settlement-free peering is a thing, but it is really difficult to set up and maintain those relationships.

The limitation isn't the NIC, it's the processor (and possibly storage). The 10 and 40 Gigabit NICs exist but processing them on a conventional PC is hard.

It is, just about, possible to perform actions on every packet in a 10Gb stream on an x86 machine. You have to use a userspace stack, handle packets across multiple cores, and be VERY careful with what you are doing so you don't do cache misses. At 10Gb/s you're talking only a few hundred clock cycles per packet - anything that doesn't work as planned causes massive backlog.

Now try serving (dynamic) HTTP to that.

Because 665 is a bigger number than 10 and 40

:-)

Akamai has ~140 000 servers around the world. Attack was probably spread around many locations that's why you don't see any report on mailing lists.
The ddos attacks seem to be getting larger these days.

I've recently seen a ~200 Gbit/s hit us.

Does anyone have good resources around mitigation? I was looking at the BGP flowspec but was hopefully that someone might have come across other tactics?

>The ddos attacks seem to be getting larger these days.

Consumer bandwidth is increasing.

tl;dr Akamai was hosting his site pro bono. His site was being DDOSed, which cost Akamai a ton of money, so they kicked him off since they were literally only losing money on the deal.
Why would it cost Akamai a ton of money? If they already have the infrastructure and automation in place to mitigate such attacks, it wouldn't be a lot of work and would not require additional resources to mitigate this attack? And as others have said: isn't this bad PR?
all those botnets ddossing Krebs = less DOSses for the rest of us = less new clients for Akamai ;)
You still have to pay for bandwidth and if the attack began impacting SLAs and paying customers, that's bad for business.
> tl;dr Akamai was hosting his site pro bono

Let us not permit companies to co-opt language for their benefit.

If it was genuinely pro bono ( lit: for the public good ) then they would have taken all steps possible to keep the site online since the public good was served more by having Mr Krebs online than not.

However, in this case they were hosting him free-of-charge because it was good publicity for them. That's a very different scenario.

"Please make sure to secure your own oxygen mask before helping others"
Krebs is the one who used the term "pro bono".

Besides, pro bono isn't literally "for the public good", it is literally "for good".

Finally - that is a ridiculous standard to hold everything categorized as "pro bono" to. Law firms oftentimes take on cases/clients that can't afford their services, pro bono. Because they call it pro bono, does that necessitate that said law firm should continue to fight all pro bono cases in court until either A. they win or B. they go bankrupt? Of course not.

Well, to be fair to the parent poster (and without going deeper into the merits of this side-argument), I'd note that the expression "pro bono" as generally used in English is actually short for "pro bono publico" which does in fact mean "for the public good".

Law firms (and other professional services firms) call it "pro bono" when they use their specific skill-set to provide their services to those (e.g. the indigent) who couldn't otherwise afford them.

In that example, it's the fact that the indigent can get access to quality legal representation which is itself considered the "public good".

I get that. My point about law firms is that most of them won't take pro bono cases all the way to the supreme court (or equivalent). i.e. there is a limit to how much manpower they are will to expend on a charity case.

In the same way, expecting Akamai to provide free service to Krebs until the end of time because it was referred to as pro bono (even if it was them, which it wasn't) would be silly.

tl;dr - Akamai provided a service that could be seen as publicly beneficial. As long as they were providing free service to Krebs, they were doing something that was arguably pro bono. Them no longer choosing to provide that service does not retroactively detract from its public benefit.

Isn't this the point at which Cloudflare is supposed to gain a handful of PR points for putting him back online, pro bono, and then doing a write up on how effortlessly they handled the bandwidth with eBPF?
Unfortunately, Krebs has (correctly) repeatedly attacked Cloudflare for sheltering most of the most prolific DDOS attackers. I doubt that's going to happen.
Care to fill in the details for me? Do you mean to say the most prolific DDoS attackers work for Cloudflare? Or that their network somehow (?) shelters them? What do you mean exactly? This sounds interesting.
Cloudflare hilariously acts as a reverse proxy for booters and does not respond to abuse reports against them.
I can confirm that this is true. "Hilarious" is not the word I would use to describe it, "potentially criminal" I think is closer to correct.
I'd call it business savvy.
They seem to think they have firm legal ground.

Keep in mind that most reverse proxies will do the same thing. The only difference with Cloudflare is that you don't know the destination IP.

(comment deleted)
Why send cloudflare the abuse report rather than the police who are paid by the government to investigate crimes. I have little understanding for this current trend of letting suspected criminals go un-investigated while all focus is on private third-parties that is held up to act police, judge and jury.
Because one would think that it would be in CloudFlare's interest not to harbour criminals on their network. This logic seems to work almost everywhere else on the internet, including privacy friendly hosters in iceland. It's mostly just CloudFlare who replies to every abuse report with the same "WE R A REVERSE PROXY", no matter what the actual issue that was raised with them was.

If they were any smaller, their IP ranges would just go into the rogue-isp-blocklist, and that would be the end of that. But because they're mixing in the criminals with their normal customers, that's not really possible.

And since I am unlikely to be in any jurisdiction that CloudFlare is in, nor do I have any chance of finding out who these criminals are because CloudFlare is protecting them, going to the police here wouldn't really achieve much.

Because one would think that it would be in CloudFlare's interest not to harbour criminals on their network.

It's also not in their best interest to take on the burden of deciding who the criminals are.

I have personally witnessed Akamai use strongarm tactics to "prove" you need their, "protection" in such a ridiculously high profile instance I am sure they have no shame.

If this is a CloudFlare Vs Akamai attack Krebs isn't saying, but I would put dollars to doughnuts it is.

While I'm open to believe you if true, you need proof, links to affected people giving details, etc... Not your personal anecdote that may or may not have happened the way you describe it.

Otherwise your post is merely an unsubstantiated personal attack against akamai.

I've talked about it before in lots of contexts and no one believes me anyway. Why bother?
well if you've never offered factual proof to back your "I've seen things" statement, that's not a surprise. If you've ever given proof to back it up, I would love the corresponding link.
Cloudflare protects everybody who signs up, including control panels for "booter sites", which are web pages where you can allegedly buy a DDoS attack for an hourly rate.

Kind of like the police protects gangsters from getting shot by other gangsters, but you would really like them not to do that, so that the gangsters can just shoot each other.

In this case, Brian Krebs tried to convince Cloudflare to kick off the booter sites, so they are unprotected, and can DDoS each other. Cloudflare didn't put any effort into that idea, and now he's apparently angry that he didn't get through to them.

That's not called being "angry". That's called being principled. Other things that are considered principled include posting your opinions with your name on them instead of cowardly resorting to a throwaway account like you just did.

Cloudflare is not the police. They're a private organization that makes a profit from offering "protection" for people getting DDoS attacked. They enable the people doing the DDoS attacks by protecting their booter sites (https://www.google.com/search?q=ddos+booter). That's called a racketeering operation (https://en.wikipedia.org/wiki/Racket_(crime)), and that's illegal. There are laws against it. Just because our crappy government is too incompetent to file charges doesn't mean it isn't illegal.

If Cloudflare thinks they can foster criminal activity through their network because they're running a juiced up nginx proxy, they're wrong. The "slippery slope" argument is absolute nonsense. As Krebs himself pointed out, they already remove sites that are hosting phishing attacks and malware.

Cloudflare, it's time to do the right thing here and stop protecting DDoS booters. Your policies are helping to damage the internet and censor people, whether they're illegal (they are) or not.

They enable the booter's free speech; they aren't enabling their actual attacks. If comparison to police doesn't suit you since they're special, think of doctors.
You're seriously comparing DDoS attack markets to doctors?

It takes an intense contortion of the concept of freedom of speech to apply it to this malicious and illegal activity. I guess under that logic it's also okay for a personnel security company to host (or hide behind their nginx server) a hitman-for-hire marketplace too, as long as they're not the ones doing the actual killing?

DDoS attacks (and their store fronts) are not about freedom of speech. They are, always and everywhere, about the suppression and censoring of speech through violence. Protecting them means that you are protecting violence.

But if you're adamant on this being free speech, fine. Where's the free speech criticism for Cloudflare shutting down those phishing and malware distribution sites? Why are DDoS attack sites magically different and deserving of freedom of speech protection?

Honest question -- how do you feel about someone downtown with a bullhorn, calling for terrorism or some other malicious act?
Cloudflare will obviously respond to law enforcement requests of what the origin server is. Krebs is not law enforcement, and neither are other DDoSers. What is your problem?
Let's say I'm running a site. It gets DDoSed. Almost all of the booters are behind Cloudflare. How do I proceed here? Call the local police? Email abuse@fbi.gov?

Most people just panic and start paying Cloudflare for protection. Pretty much for the same reason that most people pay into protection rackets instead of reporting them to law enforcement, because they're afraid and their livelihood is at risk.

Cloudflare also has this wonderful policy of forwarding abuse reports (with information on who reported) to the booter site in question. You can imagine the consequences of that from what just happened to Krebs for doing reporting on them.

Let's say CloudFlare didn't exist and you got DDoSed. Now what do you do?
I'm not saying that Cloudflare or DDoS mitigation shouldn't exist. I'm saying that should not protect sites that are doing the attacks that they profit to defend against.
My point is the traffic isn't coming FROM CloudFlare. When you're attacked, there's no way of knowing who is attacking you. Your recourses are the same even if CloudFlare wasn't protecting the brochure/control panel websites of the services.

If you are being DDoSed. What do you do? Call the local police? Email abuse@fbi.gov?

It's not a "brochure", it's how they meet their customers and take payment from them for their attacks. It's how they make it so anyone in the world can launch a 100Gbps+ attack in 5 minutes for $20.

If you get DDoS attacked, you panic and look for expensive DDoS mitigation, or you go out of business. Legally, enforcement for the specific attacker is almost impossible. Cloudflare both knows this and benefits from protecting it. They realize that customer connection is critical to the system functioning and yet continue to defend it.

> Let's say I'm running a site. It gets DDoSed. Almost all of the booters are behind Cloudflare. How do I proceed here? Call the local police? Email abuse@fbi.gov?

With or without Cloudflare in the mix, how the heck would you go about making the connection between a flood of traffic from a large number of IPs, and any particular booter site? I don't understand how taking Cloudflare out of the mix helps you stop the DDoS.

As an aside, the FBI is indeed interested in investigating large DDoS attacks. Contact your local field office to see if yours qualifies! :-)

Cloudflare realizes that the status quo makes it hard to prove standing to sue, and that's a large part of what allows them to get away with it. But if you go to Google right now and search (https://www.google.com/#q=ddos+booter), you will find that basically all of them are behind Cloudflare. They have spent an enormous amount of time brainwashing Silicon Valley into thinking that this is a free speech argument (as evidenced by some of the absolutely ridiculous comments in here comparing DDoS attackers to unpopular speech protection or making absolutely shameless comparisons to whistleblowers like Aaron Swartz).

DDoS-for-sale sites are not "free speech sites", they are for-profit criminal organizations engaging in the violent censorship of people that are too poor to afford proper DDoS mitigation or that want to control the privacy of their users by managing the SSL certs.

The booters aren't usually as powerful as these 600Gbps+ monsters, but they're quite adequate to wreck almost any network for a long time (most IP transit hookups for racks are 10Gbps or less, these attacks can be well in excess of 100Gbps), requiring you to spend exorbitant amounts of money to protect your site against what are essentially bored high schoolers with a spare $20, your competitors, or whomever. That money is then dumped back into the system, allowing the attackers to build even more sophisticated and powerful infrastructure, leading to worse attacks like the ones we're now starting to see. Krebs was one of the people to document this trend, and now his site has been censored off the net by the same people he was writing about. Why is it so surprising to everyone that he's avoiding Cloudflare?

If you're getting DDoS'd right now, and you want to sue the booter that is doing it, how would you know which one to sue? Cloudflare obscures the origin IP because it's a reverse proxy. But even if you know the origin IP, that's not the IP the DDoS is going to be coming from. So how does one match up an attack with a specific booter website?
As I just mentioned, Cloudflare realizes that the status quo makes it hard to prove standing to sue them or to go after the attackers, and that's a large part of what allows them to get away with it. How is the FBI supposed to conduct an investigation here? They're not going to be able to get subpoenas for every single DDoS booter behind Cloudflare (one group has documented over 200 of them).

I recognize that it's impossible to eradicate the problem 100%, but by driving it underground, you can dramatically reduce the amount of it by making it harder for them to conduct their business. Cloudflare could do this in a day if they wanted to, instead they sit behind a "free speech" argument waiting for someone to force them to cut it out. Don't say I didn't warn you if the government comes in to change the liability laws to prevent this sort of behavior in the future. Nobody's going to defend DDoS spam packets from criminal botnets as "free speech" when they're preventing all speech from occurring.

< . But if you go to Google right now and search (https://www.google.com/#q=ddos+booter), you will find that basically all of them are behind Cloudflare.

Search Google? So should Google be delisting these sites?

That would be fine and all if Cloud flare wasn't removing other Internet threat from their networks (phishing, malware,...).

They remove them all, except the one whose threat they benefit from (cloudflare has a direct interest in the ddos threat being as big as possible).

Claiming they are protecting their free speech is a load of bollocks.

Both the phishing and malware attacks would be performed over CloudFlare's networks, while the DDoS attacks would not.

What happens over CloudFlare's networks in the case of DDoS providers would essentially be the agreement of a business contract.

>They enable the booter's free speech; they aren't enabling their actual attacks.

The attacks are paid for and managed by the customers through the web portals that run behind CloudFlare. How is that not enabling the attacks?

One could imagine management and payment via phone calls or snail mail. The actual attacks couldn't happen like that.
Free speech? As an example, criminal conspiracy and solicitation of murder are both crimes of speech, and claiming a "free speech" defense will just make a judge laugh at you.
Yes, but that's a decision for a judge to make, not a private company.
It is totally a decision for a private company to make. The operative word here being "private". Private individuals have freedom of association; they cannot generally be compelled to associate with people they don't want to.
So if someone hosts a site for selling drugs but doesn't sell it himself that is free speech too? It didn't help someone who called himself a Pirate and made a Silk Road.
(comment deleted)
> they already remove sites that are hosting phishing attacks and malware.

If only...

Let me quote [0]:

> CloudFlare will forward all abuse reports that appear to be legitimate to the > responsible hosting provider and to the website owner. In response to a legitimate > abuse report CloudFlare will provide the complainant with the contact information for > the responsible hosting provider so they can be contacted directly.

So, if I report a scammer CloudFlare will forward my information to that criminal, putting me at risk. Gee, thanks!

and

> Since CloudFlare is not a hosting provider we do not have > the capability to remove content from a website.

Or to put it in the words that they answer every abuse request with:

> Please be aware CloudFlare is a network provider offering a reverse proxy, > pass-through security service. We are not a hosting provider.

Which basically translates to "We don't care, we want to pretend that we are not responsible for our actions."

[0] https://www.cloudflare.com/abuse/

FYI I was impressed by your post and just tweeted a link to it to them.
Cloudflare has a pretty simple policy. They only censor content when they legally have to, or when it's child porn. That actually opens them up to a lot of heat from people who aren't big fans of the KKK, the Westboro Baptist Church, or botnets. BUT they don't specifically allow botnets as a weird method of promoting them, it's a widely applied policy.

I would bet things would be a fair bit easier for them if they agreed to take things down which most people don't like, but from my position they are taking a very principaled stand for free speech. Are people on hn actually arguing we want more censorship on more places on the web?

Nobody in here is proposing that Cloudflare censor unpopular speech. We are asking that they stop protecting for-profit DDoS attack sites that are destroying the internet and using violence to censor people's ability to speak. That isn't a freedom of speech debate, it's a debate on the ethics and legality of defending and protecting criminal activity that financially benefits them, a timely topic now that this activity is actively threatening the ability of the internet to function for any kind of speech http://www.webhostingtalk.com/showthread.php?t=1599694 http://webcache.googleusercontent.com/search?q=cache:0uf9RIu...
I mostly agree with you but let's not take it too far.

> We are asking that they stop protecting for-profit DDoS attack sites that are destroying the internet and using violence to censor people's ability to speak.

A DoS is not a violent act. I am mostly ignorant of these things but I think attacks if this kind are a service that test our capabilities. My fear is that there might be calls for legislative actions against "DoS attacks" which would then apply to people sitting at home pressing F5.

>would then apply to people sitting at home pressing F5.

How would such a law be different from the current laws? If you sit at home pressing f5 with malicious intent and succeed at bringing a site down, you're committing a crime.

I don't know about you, but criminalizing the act of pressing F5 with any intent seems firmly on the way to Aaron Swartz-like cases to me.

What if you are just fed up of waiting for a site to reload and press F5 a number of times? And what about the (probably majority of) instances where the "attacker" is simply a person who unknowingly downloaded malware onto their computer to get free smileys or whatever?

>I don't know about you, but criminalizing the act of pressing F5 with any intent seems firmly on the way to Aaron Swartz-like cases to me.

What? Why is F5 a special case here and what on earth does any of this have to do with Aaron Swartz.

>What if you are just fed up of waiting for a site to reload and press F5 a number of times?

Did you intend to bring it down? Was it obvious that your activity would bring the site down? If answer to both is "No" then you're fine, this is how most laws work.

>And what about the (probably majority of) instances where the "attacker" is simply a person who unknowingly downloaded malware onto their computer to get free smileys or whatever?

Why are you even asking? If someone else commits a crime you're obviously not at fault...

Also, what was even supposedly wrong with the Swartz case? It was on solid ground both legally and morally, shame he never gave the courts a chance.[1]

[1]: Might as well expand on this a little so I don't get hidden by downvotes. I don't think Swartz deserved to go to prison, but given that he intentionally violated the law it's hard to argue that he shouldn't have been charged.

F5 is a special case here because it is the exact same action that a law-abiding person does. The reason I'm stressing the F5 case is because saying "Hey you pressed F5 with this motive, so you go to jail" is equivalent to thoughtcrime – you're being punished for your thoughts rather than your actions.

Now if someone is using tools specially built for DoS I don't have a a problem with them being prosecuted.

> tools specially built for

That is also a problematic definition. I recall similar arguments being made against "nmap"; should we ban nmap, or criminalize its use? I also remember when Dan Farmer was fired for simply writing a security scanner (https://en.wikipedia.org/wiki/Security_Administrator_Tool_fo...), using the same reasoning.

If you figure out how to build a 600Gbps DDoS attack with Firefox, you are correct, that still qualifies as a DDoS and you can go to jail for it already. People have been tried in court for using Low Orbit Ion Cannon before, in a few extremely isolated instances. A DDoS is a DDoS, but intent is obviously important, and you do need to actually cause a problem for there to be a crime. I think clicking reload a couple times would be a stretch here for enforcement, perhaps it's possible but AFAICT it's not yet happened.

But we aren't talking about protest with a reload macro here, these are for-profit criminal botnets. And one if them just took down the largest DDoS mitigation network in the world. Which means there aren't many sites on earth left they can't take down. Much smaller attacks have nuked Github for days. Who's next to get "freedom of speeched"?

I think a better solution is to have ISP s work together to warn and cut off access to botnet infected computers. They have the technical ability because they have strikes for copyright. Perhaps it could be a soft ban like an hour long ban or something.

But if two billion people decide to stay at home and continuously press F5, you should get freedom of speeched. I think that's the equivalent of a picket line. Not talking about automated tools other than "refresh page every second".

"I will send DDOS for $xxx, send paypal to ###@example.com"

That's the the extremely unpopular speech that you're proposing to censor. The instant you say "oh but that's different" because of the contents of the speech, you're interjecting your own opinion about that speech.

Which, actually, is fine, but don't play that off as not being speech.

At the level where Cloudflare's network isn't actually being used to send the DDOS attack itself, it's also still speech.

Cloudflare will close accounts when asked, backed by court order. The problem is on today's Internet, that's nigh impossible, which realistically means it falls to Cloudflare to interject an opinion on what's good and bad, but so far they've avoided that as effectively as an ostrich burying it's head in the sand, and so are effectively supporting many bad actors.

While I understand your position, the particular line you quote is not protected as 'free speech' because it's advertising to sell a criminal act for money ... I could be wrong.
I agree that it's an ethics problem, and a non-trivial one at that.

It seems like another problem caused by the fact that code can be data and data can be code. By which I mean, both are information. 'Free speech' implies the intent to be communicated to people, and can be considered 'data'. However a DDoS is a bunch of information with the intent of affecting the behaviour of computer systems, and can be considered 'code'.

The problem lies in discriminating between the two, given that "bits don't have colour", as explained here: http://ansuz.sooke.bc.ca/entry/23

I'm not at all sure what the right answer is, here. I'm also not 100% convinced that Cloudflare has the right approach, but I'm leaning to "yes", considering the alternative.

(by the way, you'd probably be interested in watching the youtube clip jgrahamc posted elsewhere ITT, with someone from Cloudflare saying some words about their perspective on this dilemma: https://news.ycombinator.com/item?id=12564876)

Protecting unpopular organizations is taking a principled stand for free speech. Protecting people who profit from breaking people's web services is not.

"We don't take it down unless it's illegal" is a simple policy, but to be a good policy it needs judgment as well.

Cloudflare has a pretty simple policy. They only censor content when they legally have to, or when it's child porn. That actually opens them up to a lot of heat from people who aren't big fans of the KKK, the Westboro Baptist Church, or botnets. BUT they don't specifically allow botnets as a weird method of promoting them, it's a widely applied policy.

I would bet things would be a fair bit easier for them if they agreed to take things down which most people don't like, but from my position they are taking a very principaled stand for free speech. Are people on hn actually arguing we want more censorship on more places on the web?

I wonder why. Any ideas?
I imagine he is still talking with Akamai(they did not comment after all) and expects to be back after attacks die out. Switching would burn that bridge.
Akamai was good to him by providing services pro bono, so I doubt he wants to knowingly become a PR stunt for CloudFlare.
I'm surprised that the Azure or Google Cloud teams aren't on top of this. They want tech people to pay attention to their stacks, why not host a high profile site like this to gain the respect of the industry?
I feel like Brian Krebs is a public good at this point. Would love to see Google foster a better web by hosting him!
They should, and get agreements with CDN's / ISP's to forego charges in case of a DDOS. If anyone could pioneer such an agreement, it would be Google.
(comment deleted)
You want them to push the idea that isp and other middle men networks should not be dumb pipes, but charge a different pricing depending on traffic type and intent?

Your comment may have the best of intentions, but that's how you take net neutrality out the window.

New way to get on google for free: make an AMP site. Until they start charging for that at least.
Considering they charge per GB data transfer, even if your site survive the attack you'll get an amazingly expensive bill at the end of the month.
There could be something like a DDoS insurance.
Insurance only works when you can get good coverage (lots of premiums being paid). Akamai and CloudFlare are DDoS insurance, and BK just got dropped.

You don't see insurance companies rushing in after a disaster, this is no different.

For both Google and Amazon the cheapest transfer you can have is 0.02 USD/GB, which in case of such attach gives 5.5k USD / hour, or 400k USD for three days of the attack.
That's for outgoing (egress) transfer. Incoming (ingress) transfer is free with both Google and Amazon, and a DDoS attack is mostly incoming.
You just have to make sure your infrastructure doesn't try to auto-scale up to actually handle all the traffic or suddenly you have thousands of high-powered instances to pay for...
Amazon's actual ELB IPs can handle relatively little traffic, "prewarming" is required in order to add more ELB instances--during a DDOS attack, you'll be overwhelmed almost instantly. Route 53 uses DNS round robin, which is trivial to bypass if you're planning on a DDOS attack (by targeting a specific IP). Google actually gives you an anycast IP, so they're a better option.

All that being said: the idea that only ingress traffic matters during a DDOS attack isn't quite right. If the connections are legitimate, you either need to be able to detect the attack attempts (requires expensive coordination and mitigation techniques, especially if the attack is much larger than what a single NIC can handle) or actually serve back the content (which will make your egress skyrocket).

The enterprise doesn't care. It'd take a hellluva an event to get marketing behind such a stunt:
Akamai is probably way bigger than Cloudflare, so if they don't take customer that got a 620Gb DDoS I doubt Cloudflare will.
I think the bigger issue was that he wasn't a customer (they provided the service pro bono), not that Akamai wouldn't keep a customer that got hit with such an attack.
Traditionally, journalists and celebrities getting freebies get /better/ service than regular paying customers.

I don't see why being pro bono would matter in an established company?

Cloudflare has this on their website: "600Gbps: Largest DDoS attack stopped" - https://www.cloudflare.com/under-attack-hotline/ But I suppose attacks may differ in other ways that Gbps.
If I remember what cloudflare's article said, that 600gbps attack was a reflection/amplification attack, which for them is likely easier to filter out then just large amounts of direct DDoS traffic, which is what this was evidently.
If you're under attack, try us out. We'll aim to surprise you in the most unsurprising and dull way possible, by keeping your site online throughout.
Shouldn't it be the job of the police to protect his web property. The police, or another government agency, protects citizens offline, why not online? Why do we have to rely on private entities for basic protection online? Time for an online fire department or something similar?
I'm pretty sure I've heard some peoples describe CERT/CIRST centers as "web firemen".
Not that I'm opposed to something like that, but depending on how big a target you are private security is already a must in the real world.
(comment deleted)
have things changed so much that cloudflare has enough capacity to seriously contemplate handling something that Akamai could not handle?

You've gotta get the bandwidth to your filtering servers before you can filter it. DDoS mitigation, as I understand it, is first and foremost a matter of having more capacity than the attacker.

No, it's just that Akamai (probably) didn't want a "customer" that they were hosting pro bono to affect paying customers.
It would be interesting to try out some of these new p2p website technologies like IPFS/WebTorrent with these high profile sites who are frequently attacked.
+1 for IPFS

Hosting static blogs is really easy on IPFS (and if you absolutely can't live without comments: use disqus) but the URL's are cryptic and you either need a public IPFS gateway to access the site - which could get DDoS'ed - or run your own.

Another alternative is ZeroNet but you still need to run the client to access the site.

IPFS team member here:

If the URLs are cryptic, you can use dns to make them look nicer. Take a look at the TXT record for ipfs.io, as well as the TXT record for _dnslink.ipld.io

Both of those websites are hosted through ipfs and have A (or CNAME) records pointing to our gateways. You can also access this locally if you happen to be running an ipfs daemon at http://localhost:8080/ipns/ipfs.io

At this scale it must also cost a ton of money to carry out this attack, I wonder if there's a vulnerability that we don't know about that let them do this so easily?
> At this scale it must also cost a ton of money to carry out this attack

It doesn't; you're using compromised machines to initiate the attacks, which is free to you.

You're almost never using machines you compromised. Instead, you're using someone else's compromised machines: a botnet-backed DDoS-ing service, which you rented on the open (black) market, at the going rate. Bigger attacks still cost more in such setups—whether or not the botnet nodes were free to acquire, the resulting network is still a scarce resource whose price rises with demand.

(Of course, in the very special case of Krebs, the people he is reporting on frequently are the owners of the botnets, who can of course use their own botnets freely.)

Well, no, if you are committing crime you can just use illegally pwned computers.
Quite impressive. You know your blog is good when folks will try to take down a CDN to supress what's on it. He's also had heroin mailed to him in combination with a swatting attempt before: http://webcache.googleusercontent.com/search?q=cache:gEjqPfc...
holy shit. that is terrifying.
Normal game streamers get swatted and sending heroin would be easy with the silkroad-alternative market online.
Does that in any way negate the fact that it is terrifying?
Could have been child porn.
An almost horrible, related story:

When I was in middle school and the internet was still fairly new (we had just gotten it) a classmate of mine hatched a terrible plan to get rid of a teacher he hated. He waited until the teacher was out sick one day then during the substitute's typical teaching pattern of having us "read these 3 chapters, answer questions then keep your head down until class is over" he jumped on the teacher's computer. After reassuring the substitute that he was allowed to he tried desperately to find child porn. His plan was to save it into semi hidden folders onto the computer then later on turn the teacher in for having child porn.

Fortunately this classmate wasn't able to find any and eventually gave up. But I've always remembered his plan. It's terrifyingly believable that if someone managed to get into your computer and download child porn, there is likely little recourse or way to prove you did not do it.

In this case such prove should be possible. Files creation dates would show that the files were downloaded when the teacher was absent. The dates would make it easy to associate this accident with your class and then, after asking a few questions, with your classmate.
Those dates can easily be altered by someone knowledgeable though.
And easily sold to a jury as proof positive after they were altered. I've feared for a lot time that this type of thing is currently in the government arsenal of threats and weapons.
> Files creation dates would show that the files were downloaded when the teacher was absent.

But file creation dates are easily changed by a skilled hacker.

And also by the teacher. So maybe he altered them? That's what the DA would sell to the jury...
>Files creation dates would show that the files were downloaded when the teacher was absent.

And what if the teacher was not on leave or sick, but just in the school's cafeteria, or briefly in another class, etc?

Good luck proving anything with the dates, especially after several months, where nobody remembers who was where at that random day in the past.

This is basic criminal defense stuff, actually. "What time did you go to the cafeteria? Did anyone else see you there?" Then talk to the people who he says saw him.

Are there any cameras in the school that might have captured his visit to the cafeteria? Review the footage.

How did he pay for his food? Did that create a record that can establish a time and location?

Are there any witnesses who saw the kid sit down at the computer? Like the substitute teacher, for instance?

None of this is unique to child porn cases. Establishing or disputing time and location is basic trial strategy. All a defendant has to do is create reasonable doubt, not conclusively prove innocence.

>This is basic criminal defense stuff, actually. "What time did you go to the cafeteria? Did anyone else see you there?" Then talk to the people who he says saw him.

And they'd remember they saw him after 6 months, and even more so that they saw him leave in 15:20 instead of 15:10, because?

>None of this is unique to child porn cases.

No, but all this make "I didn't change them because filestamps in file are in an hour I wasn't there" difficult.

Heck, the teacher himself will probably not remember where he was at the time the timestamps show...

The real problem is that the minute it was found the teacher would probably be fired, or at a minimum put on administrative leave, the media would catch word of it and publicly tarnish him, parents would demand swift action, and before the investigation even had a chance to get going the guy's life would be ruined. In the end, even if he was found innocent, the lasting damage would done.
Honestly, even if you can prove you were framed 100% and are found not guilty, the media has likely already run several stories regarding the teacher who's computer had child porn on it. The school will be forced to fire them. The school board will likely bar them from ever working in another school in that city again. When you Google that person's name the top categories will likely be regarding their trial / arrest.

Once an accusation of child pornography (creation or possession) is put out there if any of it gets exposed to the public by the way of the media (and it will) that person's life is seriously ruined even without prison time.

The problem seems to get worse, too and I certainly don't have any good ideas regarding it. Except maybe re-tooling a new search engine that somehow can take context / validity into account but that's exceptionally difficult to do. And even then if someone gets their news or information from any other source you're still screwed.

Aren't CP laws set so that it doesn't really matter if you did it, the possession by itself is a crime, no need to prove intent?
There are 50-60 parents in any given class. A fair number of them can be expected to make trouble no matter what the police does.
Yes, and every image is a separate charge.

Because think of the children.

Possession and distribution of child pornography is perfectly legal if you're the FBI though.

There's a further nuance there as well. Say you work for a site that accepts UGC images (like a major social network for example). It's required that you have a team who works with law enforcement to handle investigations / takedown / preservation. Anyone who works for said team unfortunately thus has to see those images and have them, at least temporarily, on their machine, but there is zero legal precedent to protect them. It's a legal grey area and thus I have and continue to recommend anyone in such position to refuse to risk it.
You have the defense of "we are specifically required by law enforcement to do this" which is pretty strong. It's no different from a locksmith being asked by a cop to break into someplace the cop needs to be.
I think the psychological trauma would be a much bigger concern for most than some vague legal threat.
This was my thinking as well. On a huge social networking site I would imagine this comes up an awful lot and to have to see those types of images along with other abuses has got to be difficult. I'm also curious if you see enough of them how it changes your mind and if there have been any studies on it.

Fortunately I know you can integrate with that hashing project (I forget what it's called) where they generate a hash for known images of child porn so those can at least be removed automatically but I don't know how much of a percentage that catches.

>Yes, and every image is a separate charge.

Each video frame is a separate charge...

Hang on, just to clarify: if someone else puts a file on my PC without my knowledge, without me ever having seen or opened it, he could call the police on me, and it would get me convicted?
> Hang on, just to clarify: if someone else puts a file on my PC without my knowledge, without me ever having seen or opened it, he could call the police on me, and it would get me convicted?

Yes. In such a case, a presumption of innocence would be true in theory but of no value at all in practice. Such an attack could even be carried out remotely -- an attacker could compromise a machine remotely, then plant incriminating evidence on the compromised machine (i.e. child porn, terrorist literature, drug-dealing evidence, etc.), then alert the authorities.

This plausible scenario is another reason to vigorously protect one's computer against external attacks.

I won't say that it doesn't happen (bad outliers always exist); however it would be very unlikely you'd be convicted for it.

A few years ago when I worked in computer forensics there was this big myth that if you went on porn websites and one of the images was underage you'd get done for it. However intent is a big part of law and so there would always have to be something along with "just an image" showing some intent to have obtained and viewed it.

There was a person in Finland mass scraping the alt. newsgroups for porn, and they got some cp on their machine inadvertently. Conviction, publicity and a ruined life promptly followed.
If someone else put 10 kilograms of cocaine in your house without your knowledge, without you ever having seeing or opened it, he could call the police on you and you would get convicted.
Whatever happened to reasonable doubt, innocent until proven guilty, all that stuff?

What kind of court routinely convicts people who didn't actually do the crime?

> who didn't actually do the crime?

Possession of narcotics is a crime, isn't it?

something being on your property is not the same as possessing it
I'm not sure if you're joking or not—but I'm answering anyway.

Surely you don't think you should be the one going to prison if someone broke into your house and planted 10 kilograms of cocaine under your mattress?

I'm being serious.

> Surely you don't think you should be the one going to prison

I don't think you "should" go to prison, but I'm saying that's likely how it would go down. So it's analogous to the files on a computer situation.

(comment deleted)
Possession seems to be a very nebulous concept in English and US law (probably because of their common law systems).

In contrast, German law defines possession as "having effective control" (with some more nuance obviously). Possession is also entirely different from ownership in German law. I can not control an object I have no knowledge of. If you place an object in my house, I only gain possession of it once I discover it.

I think this is one of the cases where the civil law approach of rigorous definitions is clearly superior to the common law approach of establishing precedent.

The thing that happened to "all that stuff" is called the War on Drugs. It blended in to pretty seamlessly into the War on Terror.

The word "routinely" might be contentious, because we have almost no numbers on it and the law has a way of making things "true" despite reality, but the kind of court you're referring to is just called a court.

You might google the Innocence Project, if you're interested in this sort of thing.

The comments here are overselling the situation. Yes possession of certain child porn images is a crime regardless of intent. However prosecutions are no different from any other crime: they must prove the crime before a jury of your peers beyond a reasonable doubt. There's nothing automatic about it.

This topic always drives hyperbole here on HN and I'm not sure why. Investigators and prosecutors don't waste time trying to entrap innocent web developers. They are kept plenty busy by people who are actually making or distributing child porn. Source for that: I know someone who prosecutes child porn cases. He is kept incredibly busy with obvious scumbag criminals.

Law enforcement's ultimate goal is always to walk the chain of possession back to find the folks who are actually making the imagery--who are actually abusing kids. That is why there is strict liability for possession. It gives investigators a lever to flip distributors to help find the sources.

In the US it's not obvious that an anonymous tip would establish probable cause for a search.
They don't even have to call the police. Simply put it in your Dropbox, which can be done from many devices (it's Dropbox after all).

Dropbox (and all other "cloud storage" providers) actively scan for CP material and will turn you in to the police automatically.

I need a link to an instance of this before i will believe it
I don't know about Dropbox, but I do know for a fact Facebook and Tumblr definitely do. It might have evolved since a few years ago, but they used a technology from Microsoft called PhotoDNA [0] to hash all uploaded images and match against known signatures of CP and most definitely involved law enforcement when found.

[0] http://www.microsoft.com/en-us/photodna

In the UK, somebody can send you an obscene file over Whatsapp, unsolicited, and get you convicted without you even opening it. [0]

We have sleepwalked into being a police state and barely anyone seems to care.

Police have also started trawling Reddit to look for thought crimes. [1]

[0] http://m.theregister.co.uk/2014/08/05/whatsapp_smut_convicti...

[1] https://m.reddit.com/r/unitedkingdom/comments/53y1wi/a_reddi...

From article no. 0:

"<...> police stopped them for unrelated matters and discovered the shock images upon inspecting their mobile phones"

What?? So police in UK can now ask to inspect inside your phone without any warrant or even reason?

It looks that way. Does anyone know what the UK legal position is about forcing someone to unlock their device?

I know you can be jailed here for refusing to hand over an encryption key. I don't know whether your passcode / fingerprint counts as an encryption key in they eyes of the law.

If you refuse to hand over encryption keys in court, after you'd been officially charged that's one thing (though not saying that it's justified either), but if you've been stopped for i.e. mundane traffic check and asked to hand over your phone for inspection, that's totally wrong and oppressive.
You should try reading what you post first.

1. They weren't convicted of any crime.

2. The person was charged because he racially slandered someone. Not because of a thought crime.

The phrase "thought crime" refers to opinion without action, not literally thinking. You're just being pedantic.

"Oh it's not a crime to think, just to let other people know what you're thinking". Do you seriously think that makes any sense?

1. The first sentence in the article is "Two UK men have been convicted of possessing extreme pornography". Am I missing something?

2. Why should this be illegal? Yes, it's offensive. But I don't think offending someone should be a crime.

I can't believe they both pled guilty! Is it possible in the UK to appeal a conviction if you plead guilty? IANAL, but I think their are some narrow cases where it can be done in the US, for example if you can show that your legal counsel did not provide competent representation.
It is puzzling that they both pleaded guilty. They also both represented themselves in court, so it seems they weren't getting any legal advice at all.

Perhaps they weren't entitled to legal aid? I was under the impression that anyone charged with a criminal offence was entitled but perhaps I'm wrong.

Maybe they just wanted to get the whole thing over with quickly and not face further embarrassment? It's a good example of how these types of charges could easily be used by the authorities to intimidate people. The damage is done whether they are convicted or not.

To answer your question, it seems you can appeal regardless of how you pleaded [0]. But you have to do it within 28 days.

[0] https://www.gov.uk/appeal-against-sentence-conviction/crown-...

The UK is all sorts of fucked up. You're not lawfully allowed to withhold passwords to encrypted data you're in possession of, meaning someone can slip an encrypted USB drive in your bag, you get arrested, and sent to jail for god knows how long because you're unable to give a password to a drive that isn't yours (and maybe doesn't have any password).
I'd never considered that set of circumstances before. The politicians who voted it into law probably didn't consider it either.

And that's exactly the problem: we have layer upon layer of vague and badly-written legislation, which ends up creating terrifying loopholes like the one described above.

My tinfoil hat paranoia is not quite at the level of thinking they're doing it on purpose (although I wonder sometimes).

But it's easy to see that indiscriminate surveillance combined with these vague laws create a situation where anyone could be victimised by the authorities. It's a totalitarian dictator's dream.

At least here in Germany you would have a very hard time proving your innocence.
It's even worse: The police can do it too once they got access to your computer for some other reason: Say, provoking a three letter agency.
Is there actually any nation on Earth where powerful and/or connected people can't screw you over if they really want to and if you have no other powerful or connected people on your side? (and no money, etc.)

The thing about this stuff that's really alarming is that any random script kiddie could also do this by coaxing your machine into downloading something. That greatly increases the surface area of people who can screw you. Given the abysmally awful security profile of a lot of consumer software and devices this is very plausible.

It's not a matter of connections, it's a matter of how easy it is for the police to plant evidence. And yes it's a problem worldwide.
Yes. In US law this is called Absolute Liability. Formally, absolute liability means that a conviction does not hinge on the presence of mens rea.

You're likely familiar with this concept in the context of speeding tickets. All that needs to be proven is that you were, in fact, speeding. It doesn't matter if you could not have been aware of your violation due to, say, a speed-limit sign that was blown away by a storm. If you were provably doing 55 in a 45 zone, you have no recourse.

The argument for absolute liability with speeding violations is purely practical, I believe. The reasoning is that it's not a crime, per se, so the trade-off of individual protection vs expediency of trials is deemed worthwhile. Clearly, the same is not true of child pornography convictions.

> Yes. In US law this is called Absolute Liability. Formally, absolute liability means that a conviction does not hinge on the presence of mens rea.

Actually, that's "strict liability".

> All that needs to be proven is that you were, in fact, speeding. It doesn't matter if you could not have been aware of your violation due to, say, a speed-limit sign that was blown away by a storm.

That probably does matter, since exceeding the speed limit properly posted is usually the actus reus of speeding, so even to the extent it is a strict liability offense, the absence of proper signage for any reason (except when the speed limit is either the states maximum highway speed limit or a default limit for some other condition which does not require signage, in which case notice is provided by the law setting the default for the conditions, and the sign is a reminder) makes it so that no offense occurred. [0]

[0] Also, given that states do generally have default speed limits that apply in the absence of signage, one could easily argue that the absence of signage is itself a positive indication that the default speed limit applies, making available a U.S. v. Kantor-style "good faith" defense even under strict-liability principles. [1]

[1] https://en.wikipedia.org/wiki/Strict_liability_(criminal)#Un...

Thanks for the clarifications. What's the difference between strict liability and absolute liability [0], then?

[0] https://en.wikipedia.org/wiki/Absolute_liability

> What's the difference between strict liability and absolute liability

Not a lot, AFAICT; the main differences seem to be:

(1) "strict liability" is the term used in US (and, AFAIK, UK) law (though the latter seems to refer to a criminal offense to which strict liability applies as an "absolute offense"), both criminal and tort, and

(2) "strict liability" can be either an attribute of an offense as a whole or an attribute of an element of (the actus reus of) an offense (that is, there can be a required mens rea for some element of an offense, but if there is an element which does not require any mens rea, the element can be said to have strict liability.) From what I've seen, "absolute liability" is universally a trait of offenses-as-a-whole (though that may be because I've seen less about it, and am less familiar with the systems in which the term applies.)

In the UK, possession of Child Porn is a strict liability offence: https://en.wikipedia.org/wiki/Strict_liability_(criminal)
Is it?

http://www.legislation.gov.uk/ukpga/1978/37

> 1 Indecent photographs of children.

> (1)[F1Subject to sections 1A and 1B,] it is an offence for a person—

> (a)to take, or permit to be taken [F2or to make], any indecent photograph [F2or pseudo-photograph] of a child F3. . .; or

> (b)to distribute or show such indecent photographs [F4or pseudo-photographs]; or

> (c)to have in his possession such indecent photographs [F4or pseudo-photographs], with a view to their being distributed or shown by himself or others; or

> (d)to publish or cause to be published any advertisement likely to be understood as conveying that the advertiser distributes or shows such indecent photographs [F4or pseudo-photographs], or intends to do so.

[...]

> 4)Where a person is charged with an offence under subsection (1)(b) or (c), it shall be a defence for him to prove—

> (a)that he had a legitimate reason for distributing or showing the photographs [F6or pseudo-photographs] or (as the case may be) having them in his possession; or

> (b)that he had not himself seen the photographs [F6or pseudo-photographs] and did not know, nor had any cause to suspect, them to be indecent.

There are some amendments in the Sexual Offences Act 2003, but I don't think they turn it into a strict liability offence.

That's not true. The wording of the act's relevant to CP specifically say things like "knowingly" which is well-ruled to mean Mens Rea is required.

Real-life sexual offences, as in actually attacking a child, is strict liability.

CP laws in most countries are insane, because every politician knows what it would take to defend against reinforcing the anti-CP arsenal.
that reminds me of randomRambo's story. On stream he was searching for a file on his computer, and a file with a name containing CP. Long story short, police was called, "he molests his kid on stream" (totally untrue), police came, arrested him on stream. took more than half a year for him to get declared innocent.

here is him talking about it: https://www.youtube.com/watch?v=CzdFOpRTvyU

> took more than half a year for him to get declared innocent.

Even if declared innocent, being charged on suspicion of possession of such porn is a life-long stigma that never wears off. For example, it will be impossible for a foreign citizen to obtain a US 'ESTA' visa waiver after such suspicions.

The google-cache page doesn't seem to be loading right for me - It's still trying to pull off extra content like css and images from the regular site. That said, I found the article in way.archive.org, and it has images and formatting intact:

https://web.archive.org/web/20151115154842/http://krebsonsec...

Thanks for giving a link to this post!

Oh, that's a much better version! Thank you. I thought I linked to the text-only google cache, but perhaps not.
Wow. Imagine if he hadn't gotten access to the forum and preempted the police call.
I wonder: Is he fluent in Russian? I doubt google translate could help you out with that.
Actually yes, he taught himself some Russian, from what I remember.
If you're going to be reporting on his beat, you have to be at least somewhat fluent.
That's an interesting read but I'm really not sure why Krebs posted the real name of "Fly's" wife. She doesn't seem to be involved in her husband's activities at all, so what's up with that?
"fair game"? The Russian hacker did the same with him as he posted copies of Krebs' credit report, directions to Krebs' home and pictures of Krebs' front door.

Plus, threatening to kill Krebs' wife.

But since we're talking about ethics here: Sure, now that the hacker faces 30 years in prison, he's not short of probably sincere apologies. I could really believe that he now has changed his view and accepted his guilt. It makes me ponder the thought of if I were Krebs' to not only feel sorry for the guy but (if it were legally possible) to dismiss the charges against him.

Consider the much more likely outcome of the hackers' plan: That it worked. Would the hacker had the same sense of guilt then? Or the same sense of forgiveness as Krebs or me seems to have? Maybe. We'd never know unless he did. It's more likely, he would have enjoyed Krebs' ruined live. Maybe even continued to threaten his wife and family. Just for the fun of it.

Because doing that in the anonymity of the web makes it easy to misbehave in ways no one ever would in front of the public eye and even less in the eye of his family and friends.

I tend to be a bit skeptical of the sincerity of apologies first made after the perp is caught.
If you ever press charges against a criminal, follow through with it.

If you forgive them, they might just do it again.

I don't think he deserves 30 years either, but he should still go to prison.

Of course he's not going to get the charges dropped against this guy. Krebs should be pushing for the most harsh penalty possible against this guy so he can continually point to him as an example of what happens when you threaten him or his wife.

I think thirty years is extreme, but I think the hope is that this will serve as an effective deterrent against this kind of crap.

Fortunately in the US justice system victims do not determine sentences. That said, I agree that hackers who maliciously target an individual with disregard to collateral damage should get the maximum sentence.
>> "fair game"?

"Fair game" excuses collective punishment, then? I'm not so sure.

Also, a "wife" is a person. How do you justify retaliating against someone by harming someone else? You steal my car, I beat up your wife- hey, "fair game"?

(Not parent poster but...) No, but you steal his car, you can't complain if he steals yours.

I don't think Krebs was right to dox the hackers wife.

She's her own person. If being married to an asshat is a punishable crime then many people are even worse off than they realise.

But Krebs retaliated in kind.

I do however think that's a punch he should have pulled.

Is the wife in any actual danger from his reveal? I imagine Krebs thought no and that's the difference.
Pretty simple, you up the stakes until the person attacking you stops. You never watched ScarFace?
It might destroy their relationship by scaring her into leaving him, which is perfectly fair game. You dont deserve to have a relationship with your wife if you actively try to destroy someone else's.
Is the strongest argument - but still too weak.
>> It might destroy their relationship by scaring her into leaving him, which is perfectly fair game.

So, should Krebs have flown over to Russia and beat the shit out of her? That would have definitely "scared her into leaving" the guy.

Again: how is it "fair game" to punish one person for the actions of another?

Relatives of a victim are generally protected by the press, but relatives of a criminal are not. Right or wrong this doesn't seem outside of accepted USA norms. Other countries are another matter, where often criminals identities are protected.
> Right or wrong this doesn't seem outside of accepted USA norms.

Yeah, but the question is about right and wrong, not accepted USA norms.

I put it in quotes because I have a hard time considering it "fair" myself.

The main point here is: this anti-social behaviour of the Hacker was made possible because of his anonymity.

Anonymous towards the legal system but anonymous towards his private life as well.

It is fair to assume he kept his cyberbullying activities to himself because she wouldn't support that.

The hacker's wife was probably more endangered by continuing to be living next to a criminal.

I think this is naive and gullible. This person needs and deserves a rehabilitation attempt, which he's very unlikely to get in prison. The psychopathology of someone who doesn't merely threaten you with death, but an explicitly named loved one, is a very dangerous, damaged, malicious person. Merely being found guilty and in prison does not at all ensure this person accepts responsibility for what they did or that it's wrong.

What someone does while they're being watched isn't a good judgement of their character. What matters more is how they behave when they're not being watched.

The people he deals with don't have any rule book they follow. Likewise, it's up to him who he wants to go after. Clearly, he decided that turn around is fair game and went after his wife.

Unfortunately when you deal with criminals, you can't really justify any behavior on either side since they are both operating outside the bounds of the legal system. Outcomes are mooted in the context of the world they are operating in when it's devoid of rules, honor and morality among the participants.

I don't know why he did it, but there's a difference between a wife and a wife's name. There's a decent chance that even if the person isn't involved, her name is or may at some point be.

It's not difficult to open a bank account in your spouse's name without his/her knowledge, and use that account to do things he/she wouldn't do.

I tried to get to an article on Krebs' site from a Bruce Schneier blog post, and couldn't, then bumped into this post in HN.

It's a pity Akamai booted him off; on the one hand, I can understand that it would significantly impact on their SLAs to other customers, but on the other hand it's a shame they don't have a lower impact network to re-host him on, and use this as a learning lesson on how to better mitigate such DDoSs...

Is it according to terms/conditions of Akamai?
What is 'it' that you are referring to?
It would be interesting if he started writing on Medium (not saying technically advisable, just interesting). I wonder if he'd ever consider trying that.
Why would it be interesting?
To see how Medium would deal with massive DDoS attacks and whether they'd kick Krebs off.
Why would he give away advertising dollars?

Also, medium is bad. Everyone now thinks if you publish something on medium, the writing is suddenly a masterpiece.

Who profits from this attack?
Someone's ego.
LOL LOOK WE TOOK HIM OFFLINE HAHAHA HIS LIFE IS OVER
Well, if your site needs to be online and processing payments so your company can pay salaries and rent, a week (or month) long DDOS would basically end things for you... Everywhere I've worked, a DDOS like this would basically be an emergency.
Not many companies can go a week (let alone a month) without revenue.
I'd love to learn more about these botnets. I wonder about things like What's the average time that a compromised computer stays in this net. What is the typical computer (grandmas old PC running XP). Do the ISPs ever get involved to kill bots running on their networks?
for the most part ISP are not set up to defend or stop DDoS from within their network. Do you go to a road building company to stop car thieves?
If that road building company also collects tolls and controls access to their roads you sure as hell do
Well that's the state. His analogy is like saying 'why would your fiber splicer police this.'
no you don't because that's a bad analogy. We do however go to states and could ties to protect the infrastructures they paid the contractor to build.

ISPs are uniquely situated to stop this kind of ddos because the traffic originates from IPs they don't own. The traffic has a spoofed from address. And as a rule, the ISP.should only need to send traffic out of a neighborhood from the block of IPs that is assigned to that neighborhood. You can put a filter on every switch or even every interface allowing only traffic from the IP or IPs on the other side of the link to send traffic. A company like Comcast could make it default part of account setup scripts. If everyone did that, these would disappear over night.

Aren't there botnets that use their actual source address when spamming a DDOS target? How do you defend against that type of attack?
If a bot is using its actual source address, you can block its IP at the edge (and even ask the ISP to investigate). Thanks to IP spoofing, that's totally ineffective, so instead the only way to make the attack stop is to null route the host.
How does IP spoofing even work outside of those DNS reflection attacks mentioned on Krebs' blog? [1]

> "many were garbage Web attack methods that require a legitimate connection between the attacking host and the target, including SYN, GET and POST floods."

I constantly see references relating to DDoS attacks about how IP spoofing is such an obvious trick to use but I've never seen any way to actually do it. Why wouldn't every device on the internet spoof their IP?

[1] https://web.archive.org/web/20160922021000/http://krebsonsec...

> Why wouldn't every device on the internet just spoof their IP if it was this obvious thing?

https://spoofer.caida.org/summary.php - compromise a device in one of the ASes not marked "unspoofable." Those ASes do not consistently perform packet ingress filtering.

That's not to say that DDOS attacks stop being possible, but at least they become traceable.

I appreciate the response. I had no idea things were this bad.
Thank you, I've actually been interested in the answer to this question for many years! Never was able to solve it via googling.
What do you do when you have 10, 20, 50 million bots using their actual source address? Do you just block all 50 million devices? If so, then that would be a great way to initiate denial of service on those devices.
Actually, that would be a good way to take all those insecure devices off the internet, or at least prompt someone to do something about it (even if temporarily, like hitting the "factory reset" button) before reconnecting them.

Most countries don't allow cars on the road that are unsafe due to lack of maintenance. Perhaps it's time to do something similar for internet-enabled devices that cause serious harm to others. Hold the user, manufacturer, or network operator responsible for harm caused by their lack of maintenance.

Yes there are. but using your source address means that you are confined by the upload bandwidth of your hosts, instead of some mis configured DNS server with a 1000Mbps up in a datacenter. You'd just have to work a lot harder to get the bandwidth.

to add to that, You'll defiantly get some mis configured servers with 1000Mbps uploads. And those will be really easy to pick out of the lineup. And then you'd probably be able to call the DC and say that they should block that IP at their boarder and they would probably also comply because there's a good chance that customer that was doing 110Mbps and won't want to pay for 1000.

As it is now, because the source is spoofed, you can't really take the source offline, only take the destination down to keep the other hosts in close proximity running.

With a TCP connection you can pick the source and drop the handshake, basically never start the connection. Some of the windowing can be used to make a tcp connection less of an issue as well.

Most ISPs have a clause in their Terms of Service stipulating that you won't use the service for criminal activity. I'm pretty sure a DDoS can be argued as a criminal activity. So, it is in the ISPs right to stop service to those nodes. --Yes, I understand the consumer isn't the criminal, more so they are the victim and the crime took place on their service.

Further, technologists tend to be pretty good at solving problems. I know this isn't the ISPs problem, but it is a flaw in the network, I'm simply wondering if anyone is attempting to solve this problem at the network level rather than simply building bigger caching services to protect those that pay for protection.

This was a mixed traffic botnet. There are speculations that a large chunk was from compromised IoT devices.
How many IoT devices are out on the public internet at the moment?

and for the existing IOT devices, are they the same thing, or were different exploits used for different devices?

A botnet of coffee machines, thermostats and Teslas? :D
Are there web servers or software that blacklist IP addresses that disconnects after a short time and redirects them to a static page?
Yes, but they can be flooded as well. Every system is going to have a finite capacity. And if the capacity is exceeded, the system will slow. If the capacity is significantly exceeded, it will become unreachable itself.

So if the capacity of your system is X Gbps, then it will start to have problems if the attacker sends X + 1 Gbps. And will probably be completely unreachable if the attacker sends X * 2 Gbps.

> “I likely cost them a ton of money today.”

But more specifically, whoever launched the attack cost them that money.

Also, ha:

PING krebsonsecurity.com (127.0.0.1): 56 data bytes

I would say that is a clever move, but to be honest that is the most he can do now.

https://twitter.com/briankrebs/status/779144394360381440

     @    123 IN SOA   ns1.prolexic.net. hostmaster.prolexic.com. 2016092204 86400 900 1209600 3600
     @    900 IN NS    ns1.prolexic.net.
     @    900 IN NS    ns2.prolexic.net.
    *@    300 IN A     127.0.0.1
     @    300 IN MX    10 smtp.krebsonsecurity.com.
     @    300 IN TXT   "v=spf1 ip4:... ip4:... ip6:... a mx ?all"
     m    300 IN CNAME krebsonsecurity.mobify.me.
     smtp 900 IN A     198.251.81.28
    *www  300 IN A     127.0.0.1
It might be more useful to return the IP address of whoever made the DNS query.

This could trick the computers that make up the botnet to either attack themselves on the public interface (more resource-intensive than trying to DDoS your own loopback), or even better, their ISP's resolvers (it would force the ISP to do something about it).

With the recursive nature of DNS, I imagine that could get a little hairy as the DDoS'ers would then be targeting whichever DNS servers they were using.
He should get a Facebook page and publish a copy of all his posts on it.
Interesting idea, but probably doesn't go far enough...

Perhaps he should re-post his blog articles everywhere: Facebook, flickr, tumbler, watpad, wordpress, various feedback forums, etc.

Combat a DDoS attack with a DPD (distributed publishing defense - just made that up)

That would harm his business model, which is advertising.

It could work with Facebook Instant Articles, he may even be better off using it since they source the advertising and have been out trying to poach and source quality content.

I was going to say a different version of this.

Real men mirror.

Krebsonsecurity deserves to be on git and use something like Jekyll. Mirror it instantly in a hundred different places.

Why do you have to bring gender norms into this.
I'm sorry if it came off that way .. I thought the reference was well known.

http://quotes.yourdictionary.com/author/linus-torvalds/19029...

And give Facebook ad $ for nothing?
Isn't this whole thing a bit silly? I mean what's the point? They just spend time on making him the best marketing, he'll double his audience/readers, no?
The point isn't to cost Krebs readers most likely. It's to show off how awesome this bonnet is.

I'd guess the DDoSer is jumping with joy over this news actually, because now the DDoSer can advertise his service with "I DDoSed Krebs so hard Akamai had to drop him!"

They have shown who is stronger and probably got some PR for their DDOS services.
so long for using a CDN to protect from DDOS attacks...
There are different tiers. Some people can get away with using a CDN because they aren't that big of a target.
This was prolexic, a ddos protection service purchased by Akamai.
I understand that this is burning bandwidth for Akamai, but seriously, taking into account what is at stake here, I think they need to do their share and continue to support Brian.
Why don't we switch to a distributed network with a DHT like freenet? So many benefits, including not being able to take down content via DDOS.
The DHT can still be attacked. Here's three methods:

1: Take out the bootstrap nodes. These are several nodes that bootstrap a new client into the DHT system. BitTorrent, Inc. keeps a couple such nodes. On first boot, the client registers it's DHT address and collects a few from the bootstrapping node. The client could then can traverse the network itself. By knocking out these nodes, newly started clients now have to browse the whole IP space for possible DHT clients, which is not feasible.

2: Attack the peers themselves. A malicious program could traverse the network searching for DHT peers in the same way. At first, it would only collect a large number of DHT addresses and their corresponding nodes. Once a sufficient mass is gained, each is targeted with a low level DDOS to knock them offline to further requests. Most of these peers will be homes and local ISPs, which can't effectively deal with DDOS traffic themselves. Others trying to connect to a down client will eventually remove them from their own address space for later queries.

3: Poison DHT peers. This is probably the hardest, but once complete could poison an entire network with a switch. On each of your compromised Bot machines, you make a valid DHT node. Make a LOT of these (like a Botnet). For the most part, participate correctly with the DHT network. Collect as many valid/real DHT user and content addresses as you can and host them in your nodes. When it's time to attack, prevent these valid DHT addresses from resolving on inquiry. Even better, make them go in the wrong direction and infinitely pass around requests to other poisoned bots in your ring to prevent resolution but not hang the process. This is especially useful for content attacks because it attacks the content addresses themselves.

1a. A system could be made for bootstrap links that have the addresses of a few nodes in them.

1b. When you have 10 million nodes like torrents, you can go searching random IPs. As long as many nodes bind to the same ports.

2. Sure, if you have comparable bandwidth to the entire network you can take it down. But that's a lot harder than overwhelming a single target. Nobody can send 20mbps each to millions of IPs.

3. This is the method that takes the least resources, but pretty good countermeasures can be made.

You can say the same about any network. What you are describing is an attack on the WHOLE network (such as what Bruce Shneier recently reported), not a specific file. You can try to bring down the internet, but not a site.
Here's a link to the last post from his website. Google did not appear to have this cached:

https://archive.fo/t94ve

relevant quote from that: ››Many readers have been asking whether this attack was in retaliation for my recent series on the takedown of the DDoS-for-hire service vDOS, which coincided with the arrests of two young men named in my original report as founders of the service. I can’t say for sure, but it seems likely related: Some of the POST request attacks that came in last night as part of this 620 Gbps attack included the string “freeapplej4ck,” a reference to the nickname used by one of the vDOS co-owners.‹‹
I'd err on the side of caution and consider that string a Red Herring without more evidence to corroborate the claim. It is a nice coincidence though...
(comment deleted)
This is bad PR for Akamai and a tactical error for them to boot Krebs even if they were providing free service.

To some, the implication would will be "they couldn't handle it" so why should I trust the DDOS they are heavily promoting on their site?

At minimum they should comment on the situation, at best restore his service and learn how deal with high profile clients.

Even if they tried to mitigate and quietly semi failed (like 30% packet loss) the PR would have been better. It could be that such attack takes down their entire network hard. Verisign said a year ago to us they could mitigate 2Tbps for comparison.
Akamai can easily mitigate this.

They just don't want to provide it for free.

And pretty much no one can afford to use their services @ 655Gb/s for that long unless they had billions of $.

Well a single 48 hr mitigation costs 12k/yr contact. Unlimited mitigations probably cost about 130k/yr. Either this negative PR is worth 12-100k in savings for them or they would drop any paying customer if the attack is over a certain threshold?
They might bill you that money, but on average it costs them less to provide that service to you, otherwise they wouldn't make any money.

Because we don't know the operating margins and the distribution of DDoS costs per customer, we can't infer how much this particular attack would cost Akamai.

Maybe because of the scale of this attack, relative to the business value Krebs provides Akamai? Shedding an attack of this magnitude (by dumping Krebs) was probably well worth the PR hit Akamai took.