71 comments

[ 0.19 ms ] story [ 122 ms ] thread
Increases the need for Qualys security tools, no?
No. Qualys sells primarily the tools you use to detect if you have bugs like these; by publishing exploits, they're actually giving away some of their own assets.
Just the opposite. Releasing a PoC means that /other/ security companies can add the exploit to their behavioral databases that much quicker.
For those curious... click through the entire thread with thread-next>.

You'll be rewarded with some really great discussion!

Working exploits are useful to test and iterate on to ensure that everything is patched correctly (both for admins and researchers).

/r/netsec has some good discussion: https://www.reddit.com/r/netsec/comments/3ed4fu/cve20153245_...

They sure are valuable, he never said otherwise. His point was that the PoC exploit was released before or right at the time that patches became available and delaying the exploit PoC would give people a bit time to patch their systems.
Delaying PoC doesn't preclude others from analyzing the patch and writing exploits. The only thing delaying PoC does is provide a false sense of security.
No, analyzing a released patch and then write an exploit takes time. Compare that with taking a pre-written exploit and executing it, so it has nothing to do with "false sense of security". Fact is that it gives people more time to patch their systems.
There is no basis in the statement that witholding gives people more time to patch their systems. Even if these authors don't release their PoC, there isn't any way to prove it isn't already known to bad actors. Without the PoC, it is more difficult for legitimate users, particularly legit users with custom deployments, to know if they are vulnerable and to verify that patches actually fix the problem.
There's two options available: release patch and exploit simultaneously where bad guys all over the place can exploit users easily before their patch testing or deployment happens; release patch and delay exploit release so that only the subset R.E.ing the patch and converting it to a payload can hit whoever they chose to hit.

One choice puts a lot of risk on the users of software. One choice puts way less. You're constantly encouraging putting more risk on users and exploits into arbitrary attackers' hands with the argument that one or more attackers might already be a risk. That makes no sense. The only people with a straight-up benefit from this are malware writers who don't want to work very hard converting the patch into an exploit.

Researchers should delay the publishing of exploits at least a week so people can test and deploy the patch. I keep mentioning testing because patches sometimes break stuff themselves. Doing otherwise just saves attackers work.

As a user, how do I know that the patch I applied fixes the issue, particularly given the potential for user configuration of the software receiving patches? I need a PoC so I can verify the fix for the machines I'm responsible for.
As a user, you have to choose between different risks: the risk of attackers using easily-made exploits to hit a known flaw; the risk that they issued a patch that doesn't patch. One sounds much more serious than the other. Further, you can test the patch after the exploit is released later.

Instead, you want to acquire immediate vulnerability to a large group of hackers to prevent a patch from maybe making you vulnerable to a smaller number that either has an existing exploit or RE'd the non-working patch to devise a new one. Makes no sense lol... Better to get the patch, do basic tests to ensure it doesn't break functionality, install it, monitor the system/network, and then verify the security later when the sploit is released.

Needless to say, but analyzing the patch and writing the exploit is a lot more involved than copying, compiling and running an exploit, like 90% of the attackers (script kiddies) do.

What they did here was simply irresponsible at best.

Delaying PoC raises the cost of exploiting the vulnerability; having it makes it easier to exploit and not having it means an attacker needs to invest resources recreating it. How much this helps depends on a lot of factors, but it doesn't seem unreasonable to me to allow some time for users to patch before releasing the PoC.
I wish people would actually think about issues before saying "false sense of security." It's as useless as shouting "that's security through obscurity!" without thinking about things.

If you want to get rid of the False Sense Of Security, announce that there are working exploits out there.

The rationale for not releasing the PoC is that it gives legit users a chance to patch. This ignores the possibility of an attacker quickly developing an exploit as well as ignoring the scenario where bad actors already know and are exploiting the vulnerability. Not releasing PoC allows users think they have more time to patch than they actually do. This is a sense of security that is not actually derived from being secure in any way. What else should I call it?

This also prevents users from being able to test their machines vulnerability after patching to verify that the patch remedies the problem in their installation. Again, apply the patch, hope that it works in my installation (with my config options) is a piss poor security policy.

> Not releasing PoC allows users think they have more time to patch than they actually do

This assumes a bunch of stuff about the psychology of everyone else that you cannot possibly know.

You are making decisions for other people because you think you know their business better than they actually do.

> This is a sense of security

You are not a telepath who can read people's minds.

Yes, and my argument is that providing a working PoC along with a patch has at least these immediate benefits:

1) It helps admins verify that the patch was successfully applied.

2) It gives security researchers more surface area to evaluate when looking for similar bugs or bugs in the patch.

I'm not asserting that there are not negative aspects, but in the majority of cases waiting for some indeterminate time before releasing a PoC or not releasing one at all is net negative.

I agree with Leif here. Sure releasing a PoC serves a purposes even in these cases, but this is just idiotic. One cannot expect outsiders to patch their systems this quickly. I would release without a PoC for a few weeks, maybe a month depending on how widespread the exploit is believed to be, to give everyone enough time to update. Humans need sleep , and some patches need downtime after all. Perhaps a tiered release would be best? Release a PoC to a few trusted partners to verify and pen-test the new patches before giving a full-disclosure. If there's something I'm missing here I'd love to hear it, cause frankly I feel like I'm just stating the obvious here...
> If there's something I'm missing here I'd love to hear it, cause frankly I feel like I'm just stating the obvious here...

Likewise — I'm very much a n00b sysadmin, but I'm having a tough time understanding how giving people a buffer isn't desirable.

1) It helps admins verify that the patch was successfully applied.

2) It gives security researchers more surface area to evaluate when looking for similar bugs or bugs in the patch.

3) A PoC not existing provides a false sense of security. Just because one is not published does not prevent attackers from creating and using an exploit (in many cases before the vulnerability is disclosed).

Again, delaying the PoC exploit code release gives people some time to patch their systems. How much time depends on the skill of the attacker and complexity of the exploit, but some time is better than no time.
And again the point of Titanous is that delaying the PoC provides a false sense of security. Saying "some time is better than no time" is the problem, you have a false impression that you have time while the fact that the PoC is not published does not mean it is not already used by some attackers. Delaying does not provide an incentive to people to update. I agree that there is a tradeoff, but it is not a simple a you present it (delaying means giving some time).
It's not a matter or "impressions" but rather the matter of chance: when a vulnerability is published you may have the chance to fix it before someone figures how to use it in a exploit.

If the exploit is released immediately, that chance becomes effectively nil.

Of course this doesn't change the fact that everybody should apply the patch ASAP, but delaying the exploit simply adds some delay to the bad guys.

All of those make sense, but they seem to focus mostly on the most skilled of both groups.

I don't have any useful numbers, but that seems to leave many novice sysadmins vulnerable to script kiddies simply due to not being around right at the patch release time.

I could be wrong, though. And even if I'm not, perhaps it makes more sense to do things this way, but I'm not (yet) convinced.

Shellshock and Heartbleed both provide good examples of why public exploit code is a net good thing.

Researchers put Shellshock proof of concept exploit code into fuzzers[0] and managed to find five more bugs subsequently[1].

Heartbleed required a patch to OpenSSL, which is typically a dynamic library. I'd wager that without publicly available exploit code to test with many admins would have just installed the patch and failed to restart the server processes, leaving their systems vulnerable.

The solution to improving patch deployment timelines is automated updates, not having admins always online. Admittedly this is still an area that is underdeveloped.

[0] http://lcamtuf.blogspot.com/2014/10/bash-bug-how-we-finally-...

[1] https://en.wikipedia.org/wiki/Shellshock_%28software_bug%29

Interesting — I had no idea of those "side effects". Thanks for the links!
So, someone with such an exploit couldn't have done that a week or two later to give people time to patch? Plus, you've released a few, unusual examples where the vast majority of patches are to vanilla vulnerabilities in software that provide us little, learning experience. Saving attackers labor while increasing risk on defenders isn't good for security.

Time to respond is always valuable. Amateurs rushing out patches can cause as many problems as buggy software, especially if the patches are buggy.

If a patch can be made by upstream in good faith to addressa vuln before CRD, embargoing until after its users have had a reasonable chance to deploy it is a very good thing. If the upstream is not responsive, that's different. The point is not artificially creating fires and surprises unnecessarily, which put thousands or millions of systems at avoidable risk. Patches need to get out there as soon as possible when good faith is working and PoCs need to be released anyhow if upstream is unresponsive. This accountability will ensure that preventative, cooperative fixes are released as soon as possible with minimum damage. Because of a premature PoC were used to takedown life/safety or major business concerns, there will be additional fallout. Playing fast and loose and dismissing concerns of large-scale installed bases is not a strategy, it's either ignorance or hubris. Causing emergencies and putting people unnecessary risk is nearly always preventable bullshit. There are only a few cases where upstream developers are unable, incapable or unwilling to patch something in good faith that it should be released... And in some cases where the fix is tricky, there should be an occasional, mutually-agreed short deferment. Again, avoid unnecessary harm to users by applying common sense rather than unrealistic dogma "full disclosure right now, fuck everyone else."
> > I'm having a tough time understanding how giving people a buffer isn't desirable.

4) There is some fame in being the first to publish a full exploit. Not as much as publishing a vulnerability, but still.

For 1, in this case a much simpler script that verifies if newline can be added or not would have been sufficient.

For 2, not releasing a PoC could be even better. Someone else from the advisory might come-up with an even more clever angle. With the PoC released and it being so intricate, there is less motivation, so less surface area is explored in this case.

For 3, yes it's always false sense, that's why the PoC release being delayed should not influence any appropriate reactions anyway.

1) How often do patches fail to apply successfully in a way that does not generate an error message from the patch management system?

2) How many security researchers actually start doing this so soon after the PoC is released that delaying the PoC by a short time (even a day would greatly help many admins) would have any effect on them?

3) This is a good argument against a long delay in releasing a PoC. I don't see how it is relevant to a short, pre-announced, delay. It is unrealistic to expect every vulnerable system in the world to be patched at the same time, immediately after the patch is released.

1) Not very often.

2) In this case, probably none.

3) The real false sense of security in this case is the notion that the patch obscures the bug meaningfully more than the exploit code does.

The security community, like any community, loves things that increase its status.

A bunch of stuff getting compromised because Idiot Software Company Couldn't Write Secure Software and making national news increases the status of the community.

It's an uphill fight against basic human group dynamics.

So unfortunately the people that you as a sysadmin are protecting against don't care about your need for a buffer to update systems. Once the patch release is out there, the exploit writers know the target and can just create the exploit quickly without a PoC.
I think the fundamental misunderstanding is that once a patch is released, you've given hackers a roadmap to understanding the exploit. At this point, there are essentially professional dev teams waiting to build out exploits. From what I've read, lack of a full poc delays hackers by hours.
While I certainly think it's true that the "real deal" would jump on any patches put out and reverse-engineer the exploit with relative ease, I expect the actual hack scene to have the same distribution of skill (if not more skewed) than the regular programming world. For every Carmack there are 10 me's, and for every professional exploit builder there is an army of script kidies. Look at how widespread Shellshock and Heartbleed exploits became after their disclosure, I don't think there are that many attackers capable of making such an attack themselves without a PoC.
True -- but skilled engineers aren't necessary. The exploit builders sell their wares. I read recently -- unfortunately I don't have the link handy -- that these exploit toolkits rent out for eg $5k/mo.
No PoC was released with the Heartbleed disclosure, but many were published within hours by others (including myself). And this was a good thing, without them admins would be unable to verify that their servers were patched successfully.
At a company where I worked previously one team only noticed one of their systems hadn't been correctly patched against Heartbleed because there was an exploit available that they could use to test.

Who knows what would have happened days/weeks/months later if that hadn't been the case.

Hours can be really useful, especially if they can be scheduled in advance.
one cannot expect to keep using shitty software with backdoors built in. FIX IT, OR WE'LL TELL PEOPLE. the date was prearranged... fix it sooner, or better yet DON'T WRITE BUGS IN THE FIRST PLACE, MORONS.

ain't nobody got time for that.

That's what an embargo is and why it exists. And when it's up, it's up.

This is essentially a rehashing of the Microsoft/Google squabble a few months ago.

Maybe there should be tiered embargoes then, so that the fix is released before the exploit?
No, Google gave Microsoft X days to fix and release, and MS missed the deadline. (Assuming I'm thinking of the same issue as you.)

Google didn't release PoC exploit code the same instant as Microsoft released the fix.

The point of an embargo is to allow vendors to coordinate patches. Since the exploit clock starts ticking when the first patch is released, releasing all patches at once means that everyone has some opportunity to patch before an exploit is in the wild.

Releasing an exploit simultaneously with the patches gives people zero time to patch. In that sense it somewhat defeats the purpose of the embargo.

To ensure that test kits, catches and patches prevail. But that's only a guess.
Responsible disclosure is a courtesy, not an obligation, to criticize reeks of entitlement.
(comment deleted)
I strongly disagree, and don't follow this logic.

Responsible disclosure is intended to be ethical, the opposite is ... well, irresponsible.

I've worked with irresponsible security disclosurists before, and even those that agree to a window only to dump another small variation after the window is up - rather than working to make sure the product is as good as it could be.

Giving end users time to patch so they are not owned is quite a ethical thing to do. The end goal is less compromised systems.

Doing this at the same time achieves the opposite effect and increases the amount of compromised systems.

It's never been "we're giving you two weeks, get it done fast because the exploit is going out then", it should be "we'll give you enough time to fix this properly, and then give the customers time to update once announced".

I personally would always release when something got done early, just because the person on the other end is usually unpredictable. However, the person on the other end shouldn't assume development of the fix didn't take the full interval.

Let people pen-test after they had a chance to update to secure the systems. Having the ability to prove a fix is not worth systems being owned prior to being able to fix them. The damage is already done at that point.

I've worked a lot of CVE reports, and about half the time, the person on the other end just wants credit, they aren't really going to hang around and try to help, and often they supply terse information (i.e. no exploit PoC to the vendor) as some form of game. I'd much rather this always be more user focused, the ultimate desire should always be to help the end user, and often users are less educated and don't have dedicated security teams.

An exploit as the fruit of someone's labor is non-trivial, often holding significant monetary value. They (exploit authors) have no obligation to release it to the vendor at all, let alone for free and "responsibly". Is the unpaid labor scheme you're describing "ethical"?

If vendors want "responsible disclosure", they can simply buy exploits at market value.

Sounds great, but doesn't really work with open source software (unless I have some wires crossed). Who is supposed to pay for an exploit in (say) MariaDb?

The value to a sufficiently motivated black-hat organisation is surely far higher than the value to any one user of MariaDb, no matter how big they are.

>Responsible disclosure is intended to be ethical, the opposite is ... well, irresponsible.

It turns out the name was chosen for its connotations, i.e. to sound good. If you call it "coordinated disclosure" it doesn't come into the discussion cheating.

The points stand on their own, and I think it's useful to have a phrase that markets the practice.

I personally view it as irresponsible regardless of whether we called it "Foo Disclosure". I do view sharing the exploit on the patch date irresponsible irrespective of the term. I believe the name was chosen well, and I don't think it deserves painting with "PATRIOT ACT" type brush.

I've dealt with commercial firms as well as non-commercial firm disclosures, and really working with people who report security issues could go a LOT smoother. A lot of those experiences did not go well, but it would be wrong for me to make generalizations.

Business here on either side doesn't matter; the user is all that matters, and it shouldn't be about getting "credit", earning points, or being done with something quickly. I think a lot of people think this is a game or something, and it's really not.

Real systems and real people's data are on the line.

"Responsible disclosure" is a term of art coined by a group of vendors and consultants with close ties to vendors. Baked into the term is the assumption that vendors have some kind of proprietary claim on research done by unrelated third parties --- that, having done work of their own volition and at their own expense, vuln researchers have an obligation to share it with vendors.

Many researchers do share and coordinate, as a courtesy to the whole community. But the idea that they're obliged to is a little disquieting.

If vendors want to ensure that they get some control over the release schedules on their flaws, they can do what Google does and pay a shitload of money to build internal teams that can outcompete commercial research teams. Large companies that haven't come close to doing that shouldn't get to throw terms like "responsible disclosure" around too freely.

Although I take your point of principle, I'm not really sure I see any incentive to not share vulns. From what I can think, the options are:

1) Actively exploit - I think we can agree that providing it to someone who will actively exploit is ethically dubious, or at least can be classified as irresponsible?

2) Share with the vendor. Ethically, this is fine.

3) Do nothing with it. Ethically, this is fine, although pointless.

4) If you make software that's supposed to detect and/or block the use of such vulnerabilities, add it to the detection system. I don't think this is a 'good guy' thing to do, although I suppose it could in principle be what's best for the company.

Having said that, my guess is that this last one has the smallest amount of commercial value, since it turns what might be a few months of work into a tiny part of a bigger piece of software; it's an investment that probably ranges (when overheads are considered) from $5-50k that has no value at all unless someone else finds the same bug.

But if someone else finds the bug, they could as easily be another security researcher as a bad guy, and then you've kinda lost out.

Have I missed anything here?

In this case we're not even talking about a vendor not getting access to a bug found by a third party (though that does happen). We're talking about a series of vendors notified ahead of time who now expect to have a say in how the researcher notifies others.

The general response a lot of vuln researchers have to this kind of sniping is that it should be directed to the person who wrote code after, say, 1995 that directly edited /etc/passwd instead of rebuilding and linking it.

I recently reported a security issue to a quite popular open source project (which has at least some company support). The fix is ready but because of coordinating other security fixes into one big release it is - after almost 2 month - still not released. I wonder if this is normal? What else can you do? Full disclosure?
The actual answer is in the thread:

http://www.openwall.com/lists/oss-security/2015/07/23/19

"That's how coordinated release dates work. Instead of trying to shame Qualys for not following your arbitrary views on what is and isn't "Responsible Disclosure", perhaps you should make sure Red Hat releases patches hours before the CRD, like Ubuntu does?"

The rest is the usual rehashed discussion on this topic.

Immediately followed by a Canonical employee saying that no, that's not true:

"Ubuntu is actually very careful to not release anything ahead of CRD unless the CRD is broken elsewhere. Not saying we've never made a mistake, but as you can imagine it is quite annoying when a CRD is broken-- we certainly don't want to be the cause of that annoyance for others. :)"

> perhaps you should make sure Red Hat releases patches hours before the CRD, like Ubuntu does?"

We could end up repasting the entire thread, but the very next comment is from Ubuntu denying that behavior. Releasing early is basically violating the embargo.

"There was absolutely nothing wrong with Qualys' timing. When the embargo ends, it ends." And then later some comments about people's panties being twisted, etc. which ends the thread.

Reading the entire thing is worthwhile.

This is a local privilege escalation bug. If you're a SAAS company protecting user data and you were relying in some way on RCE not being a gameover flaw because it wouldn't give attackers root, you were probably already boned.

It's also a 90s style Unix file management bug. It's not like patches and vague advisories were going to keep the exploit under wraps.

That doesn't make it unreasonable to complain about exploits being released in advisories (though: this is a complaint that is approximately as old as security advisories), but it should dampen the outrage a bit.

They do it for personal marketing reasons. Someone else could have posted a PoC soon after and stole some of their spotlight.

I agree, it would be nice to wait a little while for people to get their patches installed. But also, people like to have a way to verify that the patch worked.

there's a lot of arguments here that delaying the patch release gives a false sense of security because we cannot prove that bad guys do not already have the exploit, or because they will develop one within hours.

if to have real security we must prove that bad guys do not have an exploit, would that not lead to a conclusion that there is no real security at all? it is safe to assume that exploits for undisclosed vulnerabilities exist. and, regarding the "several hours" argument, didn't the release of the exploit reduce those several hours to zero? instead of not being sure if the bad guys have the exploit, or if they will develop it in 1 or 3 hours, we are now certain that they have it. instead of a sense of false security, we currently have no security.

it seems obvious that little to nothing can be done about those who already have the exploit. and given the wide agreement that developing it will take a couple of hours, then wouldn't an optimal exploit release time be 1-2 hours?

and as far as i could notice, only one very downvoted comment mentions that a motivating factor for a quick release could be to be the first to break the news. certainly sounds like good PR for a security company. why is this unspeakable? :)

Please pardon what may be a naive suggestion. This isn't my area and illness had me "benched" for a long while. Is there an inverted solution?

Maybe it's worth releasing not a simple PoC, but an unnecessarily large ball of executable or testing procedure that incorporates the PoC for testing purposes, but also does many harmless, exploit-like but nonsensical things, performing only one action that matters (the test.) A hairball of mostly trash code or procedure that is not trivial to untangle, in other words.

The idea is to create enough of a mess to stop the script kiddies from quickly knowing what to modify for their own purposes, not forever; but for a time.

Just maybe it would be possible to create a hairball that takes longer to untangle or trace than just having a pro tear apart the patch and roll their own attack, ignoring the hairball. If so, providing the test/PoC/hairball is not giving the pros in the black hats any extra time. (But in any case, at least you are not amusing all the script kiddies.)

If you can make a worthy hairball, you might just want to release the hairball just a little ahead of the patch; enough to whet admins' appetites for the patch - inverting the current order of release. (Or not.)

Again, this may not be practical in all cases or any; and my apologies if that's so.

PS There may be an argument in here for providing an "unnecessarily" and complex patch, too, to make it harder to reverse engineer - perhaps worth considering.

For a particular system to actually be exploited, you need someone who (1) knows an exploit, (2) wants to use it on that particular system and (3) has access to that particular system.

A lot of the analysis in the comments here does not seem to be taking all of these requirements into account. They are particularly relevant in this case because, I believe, the exploit is a local privilege escalation for people with shell access.

Obfuscate the hell out of the PoC, then release it simultaneously.