Per the kubernetes-dev post, all affected GKE clusters have been patched for you:
> All GKE clusters have already been patched, so no action is necessary for GKE users. Users will see a "-gke.1" version for the MASTER_VERSION of their GKE clusters, which contains the patch.
I'm wondering how exploitable this is. Kube-dns doesn't listen outside of the cluster, so to you need to be able to run arbitrary code inside the cluster to make use of it.
Then it lets you escape to the dnsmasq container. Which does have a shell, but I'm struggling to think how this gets you more capability than "able to run arbitrary code inside the cluster".
It is far harder to exploit on Kubernetes then on a laptop running a dnsmasq cache attached to a starbucks wifi.
You would need to middle man traffic between dnsmasq and upstream DNS or have some control over DNS records and be able to force dnsmasq to do lookups. As you point out the escalation is also far more limited. Ok, so you have a shell in the dnsmasq container. Now what?
Yes, you probably want Lede, not OpenWRT or dd-wrt. I used lede to flash some old routers last year, and on the whole the experience was similar to my last interaction with OpenWRT only better (more evolved).
I would like to add that you can usually in place upgrade from OpenWRT to LEDE. Though making a config backup and a backup of the list of additional packages you installed is highly recommended. Beside of that just flash and reinstall all additional packages.
Also the devs of ddwrt do not take security seriously. Requests to include sane defaults for cipher list, dropbear etc. have been met by derision and an outright refusal to do anything about it.
For example, the Kong variant updates for http and checksum despite repeated requests to deliver over https. I’ve got quite a few other examples but sadly on mobile right now.
Damn, there's gonna be a LOT of home/SOHO-type routers out there that are affected by this. I expect we'll see some exploits for these devices in the near future.
Of the three RCEs, two of them are in the DHCP server subsystem, which means they can only be exploited by a device that successfully authenticates to your local network.
The other RCE is in the DNS subsystem, and can apparently be exploited by making a PTR record lookup to an attacker-controlled DNS server:
My first question is: can you induce people to make PTR lookups? I don't think there's a way to make a browser do a PTR lookup, which may limit the remote exploitability of this vulnerability.
My second question is: can the vulnerability be exploited with an A/AAAA record lookup instead? If so, that would open a very devastating remote attack vector, as a website would need only include a hidden image tag pointing to an attacker-controlled domain in order to get RCE on someone's router.
According to the blog post, that CVE (CVE-2017-14491) only allows a 2-byte overflow on Dnsmasq >=2.76, so hopefully that's enough to prevent it from being exploited effectively.
2.76 was only released 16 months ago though. There are probably tons of router firmwares out there that are way older than that.
I love it when vulnerabilities are publicly disclosed the moment they have patches. Now we all get to choose between patching them ourselves and manually managing patches for the affected software from now on, or leaving them unpatched until the fixes eventually find their way downstream to us.
The alternative is to deprive people willing to manually manage patches of the ability to protect their networks, so that everyone can wait for people who don't take that level of responsibility over their networks to get a one-line installer. That's no alternative at all.
Security people like to paint this as a binary choice, then inevitably siding with "disclose immediately". But it's not really is it?
Why do we put locks on our doors? It's not so that career hardened cat burglars can't get in - it's to prevent crimes of opportunity and small time crooks from getting an easy target.
I feel like there's an analog with software. Quietly commit fixes for the RCEs to DNSMasq. Those who care will stay on the latest version. You can even say "Fixes security issues" in the release notes. But, wait awhile to release the PoCs and detail the exploits completely.
The nation states will find a way to exploit it no matter what, of course. But you average cybercriminal will have a much harder time without PoCs clearly showing the attack vector.
I agree. I used to be the sole maintainer of a small Linux distribution for my employer (only a side-project, responsibility wise). This was a Fortune 50 company. I'd often have patches applied and new versions released within hours. It's very easy to do. Usually by the time someone else in the office found out, it had been patched long ago. And I'd like to be clear that I am not actively subscribed to any lists, I just casually find these issues by reading where other professionals write, often. And lo, we were more up to date than most other companies.
Sure, we were vulnerable for a brief window, and without a doubt there were other vulnerabilities, but we were pretty much as safe as possible, let alone for a half-person team.
You just don't leave windows broken. You don't leave the holes unfilled. If you find a crack in the foundation, you fix it. To do otherwise is not only irresponsible to your peers and employer, but to your customers.
My only criticism is that often the roles responsible for such things are unpleasant, come with little respect, visibility and/or pay. Companies should just suck it up and overpay a handful of engineers and security specialists, charged with the responsibility of ensuring the company avoids being vulnerable. Think of it as insurance -- it's worth far more than a bad press cycle or having your customers exploited and rightful loss of trust. The bang for your buck is off the charts.
Executives don't realize that it can end a company until it does. Apple gets it. Google gets it. Facebook gets it. But very few other companies do. Cyber vulnerabilities killed Nortel back in the day. Huawei got not just designs, but details about bids, key people, everything.
Maybe 5-10 years ago companies didn't realize it. I think every major corporation has started to consider that kind of scenario now. Smaller companies may still delude themselves into thinking they aren't targets, but they probably are much less capable than big companies of investing in the required preparations to prevent those attacks.
The "black swan" events aren't costless to mitigate. In fact, they are extremely expensive. And just like there can be a black swan for cyberattacks, there can be a black swan like a terrorist attack (like companies in World Trade Center Towers 1 and 2), a massive natural disaster (think businesses in Puerto Rico right now), massive electrical outages (say an EMP or simply just Enron malfeasance), etc.
Spending to protect against and mitigate all possible black swans approaches infinity and doesn't every pay off until the extremely rare but extremely impactful event ever happens. And in cyber security, you never know because your defenses prevented it before it ever started.
If they were doing as you describe they would at the very least keep their versions of Windows updated and they would enable things like two-factor authentication and TLS for their email. They don't. We're not talking expensive to prevent black swan, we're talking routine defences against white swans.
> Companies should just suck it up and overpay a handful of engineers and security specialists, charged with the responsibility of ensuring the company avoids being vulnerable. Think of it as insurance -- it's worth far more than a bad press cycle or having your customers exploited and rightful loss of trust. The bang for your buck is off the charts.
While my gut agrees with your proposal, my brain tells me that simply patching doesn't fix either the most common issues (basic fraud, phishing, social engineering, malicious insider, etc) or the most dangerous issues (infected supply chain, zero days, etc).
Companies will still pay for insurance because your proposal doesn't insure against those other scenarios, so they will see insurance as a crutch which mitigates some of the costs which come with unpatched vulns and the race to patch.
Of course, I'm not saying that you don't need to handle all those things -- you do. You have to have insurance, and anti-fraud departments, etc.
But compared to the costs associated with insurance or damage control, having a small team of security minded people that can make changes is basically free.
If the vuln was public before the patch, sure, get patches out asap. If the vuln is not public, wait to get downstream vendors to generate packages based on unrelease patches, and then release everything at once. This should be the de-facto stance for almost all vuln disclosure.
One line installers are the difference between a 3-day mitigation and a 3-hour mitigation. Sometimes the difference is in weeks.
It's definitely not about who "takes that level of responsibility" to patch manually, it's about what's operationally feasible. Dnsmasq is probably the widest deployed networking service for consumer devices. Now every consumer device running an old version, which includes Linux desktops, Android devices, home internet routers, etc are vulnerable. They even state how big the problem is in the release:
This software is commonly installed in systems as varied as desktop
Linux distributions (like Ubuntu), home routers, and IoT devices.
Dnsmasq is widely used both on the open internet and internally
in private networks.
On top of the wide-spread severity of this issue, they've included PoC code. It's like they want everyone to get owned.
How do they know whether the vulnerability is public or not? There's no way to know. Repeated discovery --- or even simultaneous discovery --- of vulnerabilities is common. People who abuse vulnerabilities don't publish them, so there's no way to be sure something isn't being actively exploited.
Meanwhile, as soon as you publish a vulnerability, responsible network owners can make integrity vs. availability tradeoffs, even without manually managing a patch. Somehow, as an industry, we've come to the conclusion that nobody can shut a service off, even in response to a lethal security vulnerability. Better that we risk our customers data than violating an SLA. That's messed up.
If nobody has announced it, and there are no known exploits in the wild, it's not public. It could be available on 0day marketplaces, but that doesn't mean it is being widely used. It's probably less likely to be widely used if it's a valuable 0day.
But this isn't about shutting off a service to protect a company's bottom line. This is a vuln affecting consumer devices. SLA has nothing to do with it. People's home networks and devices, not to mention a myriad number of embedded devices and sensors all over the world, are going to be affected by this - not corporations protecting customer data. Affected consumers have no means to patch themselves. Releasing this before vendors could generate patched packages or firmware is just monumentally irresponsible.
This is ostrich logic. With our heads comfortable buried in the sand we don't have to consider that we're routinely being owned up by vulnerabilities Google hasn't yet published (we are, by the way). Not only that, but when Google does get around to publishing a vulnerability, we're not even obliged to patch it ourselves; rather, we blame Google for publishing.
This would be preposterous reasoning if it weren't overwhelmingly normalized in our industry. It's an embarrassment for the whole field.
I'm not against people patching things themselves. But my grandmother can't apply patched source code to the firmware for her Belkin cable modem. And I sure as hell can't patch the firmware for my carrier's locked Android phone. My Smart TV is probably just fucked. If these vendors were contacted before release, they may be able to keep people safe at publishing time.
What you seem to be suggesting is that we should simply all disconnect from the internet until vendor patches can be prepared, the reason being it's better to stop any potential and unknown exploits going on now, than stop the hundred-fold new exploits that absolutely will follow its public announcement.
> If these vendors were contacted before release, they may be able to keep people safe at publishing time.
You mean if hundreds of the popular vendors were notified for a coordinated release? And you think that's not making the exploit effectively public? Even skipping possible monitoring of emails, I don't expect that the set of blackhats and employed security engineers have no common elements.
Good point. Does that mean there should be zero coordination? What about coordinating with the security teams for a few big Linux distributions? That would be a much smaller list of people, and might make it easier/quicker for some commercial vendors to get fixed once the public announcement is made.
The problem is that we don't have any laws forcing vendors to keep their products (smartphones, TVs, embedded (networking) devices up to date. It's high time to change that.
They seem to have shipped Android security patches a little early as a result, I just flashed the October 5 2017 OTA on my Nexus 6P (in spite of it being October 2 and all).
Exploit developers don't wait until the vulnerability is announced. Pretending a critical bug fix is really no big deal until after people patch only gives them more time to exploit, because nobody will upgrade until they're told it's urgent. If then.
Is there a real alternative? The patches themselves form as a notice, if anyone happens to be looking closely at the project. I'm sure a blog post widens the audience, but once the patches are public, the cat is out of the bag.
Sometimes, the oss community is able to coordinate patches through an embargo where the commits are hidden until major players have packages ready and staged. I wouldn't mind a 12-24 hour long embargo if that meant conveniently deployable upgrade patches were available when the issue is announced.
Especially for a LAN-only type issue like this - unlikely that someone trusted with the embargo would go blackhat LAN-side here during the wait, but now every script kiddie in the world has the knowledge to make a mess while we're still awaiting vendor patches?
This is just hubris. It's not just that the embargoes leak like sieves --- and they do --- but also the baked-in assumption that if the Google security team is first to publish something, they must have been the first to find it.
All I know is that I'll have to wait for the various *wrt distros to release updates, because setting up a cross compilation toolchain to build a patched firmware would take me longer than just waiting out for official updates. In the meantime it would've be swell if not every next-door script kiddie on the planet had a new toy in their hands...
It takes you longer because you waited for a disaster before figuring out how to build your production dependencies from source. If your network isn't important and the only person you're putting at risk is yourself, this is fine. But our standards for people operating networks that other people's data depends on are too low.
I don't want to single you or anyone else out about this because it's a universal problem among startups. But that doesn't make it close to OK.
I think it is unreasonable to expect every semi-public coffeeshop style wifi to be operated by someone manually compiling and curating patches for the router firmware every time a vulnerability surfaces. An embargo would have kept a lot of users safer because their neighborhood script kiddies wouldn't be able to go on a hacking spree. I don't think a 12-24 hour delay coordinating patches with vendors would matter for the case where a select few elite blackhats were sitting on the same bug, because they most likely have had the bug for months or years already.
I don't expect it of coffee house wireless operators, at all. If some startup comes into being that manages those wireless networks for the coffee shops, I will very much expect it of them.
I generally don't have an issue with the security decisions of people who don't know better. I do have a problem with people who should know better and essentially refuse to because of cost and convenience and availability concerns. Security should trump availability most of the time, but almost never does in our industry. That is messed up.
Yeah ok. In case you're still wondering why people use dnsmasq and dhcp, your regime wouldn't work well in most places where visitors expect to just connect and go online.
Dnsmasq is designed to automatically configure and provision dns and dhcp and provide a dns proxy in addition to other useful functions and requires virtually no configuration. Yes, it is for convenience.
Three of the issues would allow an attacker to take over your router from inside your LAN (not externally). Only one of the issues is an RCE that would usually be exploitable from outside your LAN, and there's some debate as to how hard that one would be to exploit. It might be as simple as getting you to visit a web page, but the published proof of concept would require something like tricking you into manually running a query via the command line or some other administrative tool.
How should/would we design routers and any Internet-connected device that we expect the users to not patch themselves? e.g. SOHO users.
Let's not get into the discussion of whether or not everything that can be connected to the Internet should be. (I'm in the "should not be connected camp", but I think my question can stand on its own without this discussion).
The second answer is also obvious: Try to avoid as many bugs as possible before shipping the product. If you include a highly exposed software like dnsmasq make sure you look for bugs in it before shipping it.
I don't remember having seen any professional security audit for dnsmasq before. Such an audit is probably an almost irrelevant expense for all the router manufacturers out there, yet none of them chose to fund one...
This is one of those times it's a shame I only have one upvote for a comment...
Open source is extremely heavily used by commercial software and hardware manufacturers and the reality is that very few of them contribute back, meaningfully, to Open source security.
The end result will continue to be bugs like this, with ever growing impact.
> I don't remember having seen any professional security audit for dnsmasq before. Such an audit is probably an almost irrelevant expense for all the router manufacturers out there, yet none of them chose to fund one...
"Tragedy of the Commons."
If all manufacturers in the space assume someone else will pay for the audit (or assume it won't get audited), then it doesn't get audited. If one does pay for it, they all get to benefit due to the nature of open source patches.
It's funny yet sad that it took Google Security acting on a bug report by a (presumably unrelated) engineer to find some of these security issues. Google has strong incentives to protect its own network (its livelihood depends on intellectual property and safeguarding data). The companies that sell mostly disposable SOHO routers have such little interest in security, yet it's literally the gateway most of us use to the internet.
> I don't remember having seen any professional security audit for dnsmasq before.
Mozilla ("Secure Open Source" [0]) funded an audit of dnsmasq [1] by Cure53 [2] about a year and a half ago. The final report [3] (PDF) lists one "medium" and five "low" issues.
It just goes to show that a "successful" audit (i.e., one that doesn't find any major/critical issues) ultimately doesn't prove much of anything at all WRT the overall security of a piece of software.
Consumer grade CPE should always be in bridge mode, with managent access needing a physical cable plugged to a dedicated port.
This is common sense to anyone following the 10+ years of atrocious security record of ADSL/cable boxes. As a bonus you get working ip connectivity (no half assed NAT ALGs messing up your packets).
Just fine? NAT at the end of the day is a glorified stateful firewall, but it's even worse because of the overhead of mapping (src, port) pairs to other (src, port) pairs and the mess of rewriting packets (which breaks in really fun way with protocols like SIP).
You can get the same protection NAT offers for IPv6 with an incredibly basic set of firewall rules, only allowing devices on the LAN side of the connection to establish connections and punching holes where necessary.
That's great and all, but even though my Arris gateway from my (not so) friendly neighborhood cable company is in bridge mode my EdgeRouter X connected to it (which is responsible for providing NAT and stateful firewall services for my network) uses dnsmasq and is vulnerable. Mind you, I'm pretty sure ubnt will have an update fairly quickly - but just because consumer DSL/DOCSIS gateways have a poor track record doesn't mean that prosumer or even commercial gear is perfect either.
I think we're basically in a worse version of the Android fragmentation / device support issue.
At least with Android, Google is forcing manufacturers to have a minimum security patch lifetime. There's no one in a position to require or enforce that among the tens (hundreds?) of thousands of home/SOHO routers models out there.
I expect we'd maybe be able to get some of the big players (Netgear, Linksys, D-Link, etc.) to agree individually to support products with security updates for X amount of time, but it's still dicey. I used to work for one of them[0], and often the people internally who own the products don't even know enough about the software running on them to even be aware of security issues. They're entirely at the mercy of the outsourced software firm for that. The margins are low enough (and price competition high enough) that you're not going to see much effort without consumer support. That is, consumers being actively willing in large numbers to pay more for a more secure product (so, that's not happening).
If we could go by a model where everything was based on a third party, like OpenWRT or LEDE, and if the appliances could auto-update in-place from the upstream package repositories (nothing run by manufacturers or the software integrators), then that might work, but the manufacturers would have a hard time supporting it, as I'd expect there to be a lot of random breakage.
I'm not sure there's a great solution here. Savvy consumers will pick a brand that has a strong track record with security (perhaps buying SMB or enterprise-grade products), or will flash 3rd-party firmware that they can maintain and upgrade themselves. The masses are pretty much lost and dependent on a commodity industry that has little incentive to pay close attention to security, and a lot of incentive (in the form of cost and pricing pressure) to ignore all but the worst security problems.
[0] This was 2004-2009, so things may have changed, but I'm not expecting much.
If we could go by a model where everything was based on a third party, like OpenWRT or LEDE, and if the appliances could auto-update in-place from the upstream package repositories (nothing run by manufacturers or the software integrators), then that might work, but the manufacturers would have a hard time supporting it, as I'd expect there to be a lot of random breakage.
> At least with Android, Google is forcing manufacturers to have a minimum security patch lifetime. There's no one in a position to require or enforce that among the tens (hundreds?) of thousands of home/SOHO routers models out there.
ISPs bundle routers with their contracts. So if their customers get attacked through hardware that they supplied they will have at least some incentive to put pressure on the upstream manufacturers.
It's well known that it's generally (and practically) intractable to find all vulnerabilities in a typical networked C program using reasonable auditing effort.
If auditing worked, we would know how to secure C programs and the world would be a much different place.
That seems to indicate “we” as a profession don’t actually know. Because the developer is always part of writing the code and thus, any process that results in C-Code has at least one insecure element.
Probably because you used the metaphor of a military invasion and used the phrase "death trap". That kind of talk will get a defensive reaction. That's nothing to do with whether you have a point. It's just cause and effect.
Vendor? This is open source software. It's not proprietary. There is no "vendor".
The reason these devices are cheap/affordable is because of the open source community. The reason the internet has made it to this current state is because of the open source community. If you want the internet to be better and safer you have to roll up your shirt sleeves and contribute in any way you can.
I find it really hard to understand the view that a company shipping a defective product that may cause great harm to users should be excused from responsibility because the bad software choices they made happened to be open source.
I suspect the consumer protection laws take similar views in many countries.
Like I said before, there are a number of available DHCP and DNS implementations in memory-safe languages. Many of them are open source. So the vendors don't even have that excuse.
What are they? I've certainly never heard of them or used them. Are they used in production or are they toys?
DHCP: isc-dhcp or dnsmasq
DNS recursor: dnsmasq, unbound, bind, djb's dnscache, maradns
None of these are written in memory safe languages, but they're the only ones I would trust because they have YEARS of testing and getting the RFCs right.
Meta: that page is impossible to read for me on Firefox mobile - if I pinch zoom, or scroll, and move left or right slightly Google "helpfully" send me to a different page.
How many millions did Google spend optimising the code that makes that page useless for anyone who wants to zoom in on mobile? Man, that, wow, I can't express how terrible that is, Google, really!?
As a double whammy, it doesn't display any content if you block 3rd-party javascript. Sure, there's a <noscript> section, but that doesn't handle the failure mode (or intentional action) of some 3rd party stuff not loading.
My experience with implementations of these protocols is mostly from embedded systems, but I was under the impression that they could and should be written with almost no dynamic allocation at all, since everything has very small and fixed sizes. To see so many overflows and other dynamic-allocation related bugs is thus a little surprising.
Also interesting to note that 3 of the 7 in the list definitely appear to be in handling of IPv6-related parts of the protocol.
Yet another high impact vulnerabilities report related to C implementation. History repeats. Software engineering still didn't come to the point to make engineering of safe and secure software easy.
121 comments
[ 5.2 ms ] story [ 159 ms ] threadRelevant posts:
- https://groups.google.com/forum/#!topic/kubernetes-dev/QWIzh...
- https://coreos.com/blog/dns-vulnerability-patched-in-kuberne...
> All GKE clusters have already been patched, so no action is necessary for GKE users. Users will see a "-gke.1" version for the MASTER_VERSION of their GKE clusters, which contains the patch.
Then it lets you escape to the dnsmasq container. Which does have a shell, but I'm struggling to think how this gets you more capability than "able to run arbitrary code inside the cluster".
Most likely I missed something. Help me spot it.
You would need to middle man traffic between dnsmasq and upstream DNS or have some control over DNS records and be able to force dnsmasq to do lookups. As you point out the escalation is also far more limited. Ok, so you have a shell in the dnsmasq container. Now what?
https://security-tracker.debian.org/tracker/CVE-2017-14491 https://security-tracker.debian.org/tracker/CVE-2017-14492 https://security-tracker.debian.org/tracker/CVE-2017-14493 https://security-tracker.debian.org/tracker/CVE-2017-14494 https://security-tracker.debian.org/tracker/CVE-2017-14495 https://security-tracker.debian.org/tracker/CVE-2017-14496
Just verified that my home OpenWrt WDR4300 is affected.
I'll try to launch one of the POC later in the day, but for now I assume worst case.
Specifically, here is where all updates are coming from: https://downloads.openwrt.org/chaos_calmer/15.05.1/ar71xx/ge...
No updates since March 2016.
EDIT: Has development has moved to the LEDE project? I don't understand what's happening in the OpenWRT/LEDE split. There appears to be a fairly clean upgrade path, though: https://forum.lede-project.org/t/upgrade-from-openwrt-to-led...
https://lede-project.org/start
tl;dr updated packages of dnsmasq are currently building and should be available soon
Although LEDE and OpenWRT expressed interest to re-merge.
For example, the Kong variant updates for http and checksum despite repeated requests to deliver over https. I’ve got quite a few other examples but sadly on mobile right now.
The other RCE is in the DNS subsystem, and can apparently be exploited by making a PTR record lookup to an attacker-controlled DNS server:
https://github.com/google/security-research-pocs/blob/master...
My first question is: can you induce people to make PTR lookups? I don't think there's a way to make a browser do a PTR lookup, which may limit the remote exploitability of this vulnerability.
My second question is: can the vulnerability be exploited with an A/AAAA record lookup instead? If so, that would open a very devastating remote attack vector, as a website would need only include a hidden image tag pointing to an attacker-controlled domain in order to get RCE on someone's router.
dnsmasq is used elsewhere, of course, but those deployments are far more likely to receive security updates.
That's very good news.
Edit: never mind, he says a CNAME answer to a A/AAAA query can trigger vulnerability. That's very bad news.
2.76 was only released 16 months ago though. There are probably tons of router firmwares out there that are way older than that.
(And most of the devices out there will never get updated even after this, just replaced one day)
Why do we put locks on our doors? It's not so that career hardened cat burglars can't get in - it's to prevent crimes of opportunity and small time crooks from getting an easy target.
I feel like there's an analog with software. Quietly commit fixes for the RCEs to DNSMasq. Those who care will stay on the latest version. You can even say "Fixes security issues" in the release notes. But, wait awhile to release the PoCs and detail the exploits completely.
The nation states will find a way to exploit it no matter what, of course. But you average cybercriminal will have a much harder time without PoCs clearly showing the attack vector.
Sure, we were vulnerable for a brief window, and without a doubt there were other vulnerabilities, but we were pretty much as safe as possible, let alone for a half-person team.
You just don't leave windows broken. You don't leave the holes unfilled. If you find a crack in the foundation, you fix it. To do otherwise is not only irresponsible to your peers and employer, but to your customers.
My only criticism is that often the roles responsible for such things are unpleasant, come with little respect, visibility and/or pay. Companies should just suck it up and overpay a handful of engineers and security specialists, charged with the responsibility of ensuring the company avoids being vulnerable. Think of it as insurance -- it's worth far more than a bad press cycle or having your customers exploited and rightful loss of trust. The bang for your buck is off the charts.
The "black swan" events aren't costless to mitigate. In fact, they are extremely expensive. And just like there can be a black swan for cyberattacks, there can be a black swan like a terrorist attack (like companies in World Trade Center Towers 1 and 2), a massive natural disaster (think businesses in Puerto Rico right now), massive electrical outages (say an EMP or simply just Enron malfeasance), etc.
Spending to protect against and mitigate all possible black swans approaches infinity and doesn't every pay off until the extremely rare but extremely impactful event ever happens. And in cyber security, you never know because your defenses prevented it before it ever started.
While my gut agrees with your proposal, my brain tells me that simply patching doesn't fix either the most common issues (basic fraud, phishing, social engineering, malicious insider, etc) or the most dangerous issues (infected supply chain, zero days, etc).
Companies will still pay for insurance because your proposal doesn't insure against those other scenarios, so they will see insurance as a crutch which mitigates some of the costs which come with unpatched vulns and the race to patch.
But compared to the costs associated with insurance or damage control, having a small team of security minded people that can make changes is basically free.
One line installers are the difference between a 3-day mitigation and a 3-hour mitigation. Sometimes the difference is in weeks.
It's definitely not about who "takes that level of responsibility" to patch manually, it's about what's operationally feasible. Dnsmasq is probably the widest deployed networking service for consumer devices. Now every consumer device running an old version, which includes Linux desktops, Android devices, home internet routers, etc are vulnerable. They even state how big the problem is in the release:
On top of the wide-spread severity of this issue, they've included PoC code. It's like they want everyone to get owned.Meanwhile, as soon as you publish a vulnerability, responsible network owners can make integrity vs. availability tradeoffs, even without manually managing a patch. Somehow, as an industry, we've come to the conclusion that nobody can shut a service off, even in response to a lethal security vulnerability. Better that we risk our customers data than violating an SLA. That's messed up.
But this isn't about shutting off a service to protect a company's bottom line. This is a vuln affecting consumer devices. SLA has nothing to do with it. People's home networks and devices, not to mention a myriad number of embedded devices and sensors all over the world, are going to be affected by this - not corporations protecting customer data. Affected consumers have no means to patch themselves. Releasing this before vendors could generate patched packages or firmware is just monumentally irresponsible.
This would be preposterous reasoning if it weren't overwhelmingly normalized in our industry. It's an embarrassment for the whole field.
What you seem to be suggesting is that we should simply all disconnect from the internet until vendor patches can be prepared, the reason being it's better to stop any potential and unknown exploits going on now, than stop the hundred-fold new exploits that absolutely will follow its public announcement.
You mean if hundreds of the popular vendors were notified for a coordinated release? And you think that's not making the exploit effectively public? Even skipping possible monitoring of emails, I don't expect that the set of blackhats and employed security engineers have no common elements.
Especially for a LAN-only type issue like this - unlikely that someone trusted with the embargo would go blackhat LAN-side here during the wait, but now every script kiddie in the world has the knowledge to make a mess while we're still awaiting vendor patches?
I don't want to single you or anyone else out about this because it's a universal problem among startups. But that doesn't make it close to OK.
I generally don't have an issue with the security decisions of people who don't know better. I do have a problem with people who should know better and essentially refuse to because of cost and convenience and availability concerns. Security should trump availability most of the time, but almost never does in our industry. That is messed up.
With djbdns I have never felt the need for it. Nor do I use DHCP. Or even ARP. At least, on computers that allow control over such things.
I guess I am missing out on the convenience, or perhaps there is an alternative selling point I have overlooked?
I prefer wired networks and to attach particular RFC1918 IP's to particular devices. Works for me.
I'm glad Arch Linux has such strict firewalls by default!
Let's not get into the discussion of whether or not everything that can be connected to the Internet should be. (I'm in the "should not be connected camp", but I think my question can stand on its own without this discussion).
The second answer is also obvious: Try to avoid as many bugs as possible before shipping the product. If you include a highly exposed software like dnsmasq make sure you look for bugs in it before shipping it.
I don't remember having seen any professional security audit for dnsmasq before. Such an audit is probably an almost irrelevant expense for all the router manufacturers out there, yet none of them chose to fund one...
Open source is extremely heavily used by commercial software and hardware manufacturers and the reality is that very few of them contribute back, meaningfully, to Open source security.
The end result will continue to be bugs like this, with ever growing impact.
"Tragedy of the Commons."
If all manufacturers in the space assume someone else will pay for the audit (or assume it won't get audited), then it doesn't get audited. If one does pay for it, they all get to benefit due to the nature of open source patches.
It's funny yet sad that it took Google Security acting on a bug report by a (presumably unrelated) engineer to find some of these security issues. Google has strong incentives to protect its own network (its livelihood depends on intellectual property and safeguarding data). The companies that sell mostly disposable SOHO routers have such little interest in security, yet it's literally the gateway most of us use to the internet.
Mozilla ("Secure Open Source" [0]) funded an audit of dnsmasq [1] by Cure53 [2] about a year and a half ago. The final report [3] (PDF) lists one "medium" and five "low" issues.
It just goes to show that a "successful" audit (i.e., one that doesn't find any major/critical issues) ultimately doesn't prove much of anything at all WRT the overall security of a piece of software.
[0]: https://wiki.mozilla.org/MOSS/Secure_Open_Source
[1]: https://wiki.mozilla.org/MOSS/Secure_Open_Source/Completed#d...
[2]: https://cure53.de/
[3]: https://wiki.mozilla.org/images/f/f7/Dnsmasq-report.pdf
The downside is that they yet again break P2P protocols, although that's somewhat easier to fix.
You can get the same protection NAT offers for IPv6 with an incredibly basic set of firewall rules, only allowing devices on the LAN side of the connection to establish connections and punching holes where necessary.
At least with Android, Google is forcing manufacturers to have a minimum security patch lifetime. There's no one in a position to require or enforce that among the tens (hundreds?) of thousands of home/SOHO routers models out there.
I expect we'd maybe be able to get some of the big players (Netgear, Linksys, D-Link, etc.) to agree individually to support products with security updates for X amount of time, but it's still dicey. I used to work for one of them[0], and often the people internally who own the products don't even know enough about the software running on them to even be aware of security issues. They're entirely at the mercy of the outsourced software firm for that. The margins are low enough (and price competition high enough) that you're not going to see much effort without consumer support. That is, consumers being actively willing in large numbers to pay more for a more secure product (so, that's not happening).
If we could go by a model where everything was based on a third party, like OpenWRT or LEDE, and if the appliances could auto-update in-place from the upstream package repositories (nothing run by manufacturers or the software integrators), then that might work, but the manufacturers would have a hard time supporting it, as I'd expect there to be a lot of random breakage.
I'm not sure there's a great solution here. Savvy consumers will pick a brand that has a strong track record with security (perhaps buying SMB or enterprise-grade products), or will flash 3rd-party firmware that they can maintain and upgrade themselves. The masses are pretty much lost and dependent on a commodity industry that has little incentive to pay close attention to security, and a lot of incentive (in the form of cost and pricing pressure) to ignore all but the worst security problems.
[0] This was 2004-2009, so things may have changed, but I'm not expecting much.
Yep, I'm enjoying my OnHub.
ISPs bundle routers with their contracts. So if their customers get attacked through hardware that they supplied they will have at least some incentive to put pressure on the upstream manufacturers.
It's well known that it's generally (and practically) intractable to find all vulnerabilities in a typical networked C program using reasonable auditing effort. If auditing worked, we would know how to secure C programs and the world would be a much different place.
> In addition to these patches we have also submitted another patch which will run Dnsmasq under seccomp-bpf to allow for additional sandboxing.
It takes a while, but the more known projects start getting seccomp patches and this will lower the impact of exploits like this in the future.
You can get good results with much smaller implementations too. See my patch to memcached for example: https://github.com/memcached/memcached/pull/275/files
This, the Broadcom wifi RCE and the Bluetooth RCE are like a concurrent land, sea and air pwning of people's personal and home devices.
The reason these devices are cheap/affordable is because of the open source community. The reason the internet has made it to this current state is because of the open source community. If you want the internet to be better and safer you have to roll up your shirt sleeves and contribute in any way you can.
I suspect the consumer protection laws take similar views in many countries.
Like I said before, there are a number of available DHCP and DNS implementations in memory-safe languages. Many of them are open source. So the vendors don't even have that excuse.
DHCP: isc-dhcp or dnsmasq
DNS recursor: dnsmasq, unbound, bind, djb's dnscache, maradns
None of these are written in memory safe languages, but they're the only ones I would trust because they have YEARS of testing and getting the RFCs right.
How many millions did Google spend optimising the code that makes that page useless for anyone who wants to zoom in on mobile? Man, that, wow, I can't express how terrible that is, Google, really!?
Edit: they have a "web version" (https://security.googleblog.com/2017/10/behind-masq-yet-more...) without that massive UX bug, where instead you can't scroll to see the table. smh
Also interesting to note that 3 of the 7 in the list definitely appear to be in handling of IPv6-related parts of the protocol.
CVE-2017-14491.pcap: https://www.cloudshark.org/captures/016080cbeab5
CVE-2017-14492.pcap: https://www.cloudshark.org/captures/fd4bfa9403fc
CVE-2017-14493.pcap: https://www.cloudshark.org/captures/1c30b56dc1d3
CVE-2017-14494.pcap: https://www.cloudshark.org/captures/657de976c0d1
Used this repo to generate these: https://github.com/google/security-research-pocs/tree/master...