> Like the White House, the EU confuses its stated goal of helping the software market work more efficiently with an attempt to control what can be in the market.
Control would make sense if governments had a clear, flawless track record of developing software; that state of perfection could be attributed (solely) to a bunch of existing government guidelines and the only problem would be that developers fail to meet those guidelines. We're not there.
> Control would make sense if governments had a clear, flawless track record of developing software;
Why? The government does not have a clear, flawless track record developing food, nor medicine, still it controls and regulates them for the safety of society.
You can't demand absolute knowledge as a pre-requisite of regulations, that's absurd and unachievable. Extremely black-and-white thinking as well.
If you want an uneditorialised take, here's what they quote:
The Administration will work with Congress and the private sector to develop legislation establishing liability for software products and services. Any such legislation should prevent manufacturers and software publishers with market power from fully disclaiming liability by contract, and establish higher standards of care for software in specific high-risk scenarios.
Windows, back in the day, was both buggy and full of security holes. It was also $99, and enabled an inexpensive computer to do near-workstation feats. If it cost, say, three times as much, that would have changed the market dynamic quite a bit. Fewer of us would have been able to afford computers with Windows, and few of us wanted bare-metal computers.
Well, you say, that's just fine, we could have put Linux on it, and the world would have been a better place. Well, could you have put Linux on it? Can Linux survive in this regulatory environment? Can it survive as open as it is, as modifiable as it is, as free as it is? What about the rest of free software, that doesn't necessarily have a large company behind it?
The second-order effects of this could completely change the software landscape. We may not like the results.
You'll have a blessed version of Linux from IBM/RedHat that specifically meets USC x.y.z requirements for use in a, b, and c industries. You'll have a supported version of Postgres, etc. etc. They'll be a version or two behind and everything else we might have to add riders on the licenses like 'not for use in the following industries' or 'for demonstration and education purposes only.'
Given that everything's getting either a WiFi or bluetooth chip in it, could we see coffee makers that are not approved for use in a commercial, health-care, industrial, or business setting?
All for curation of components, as part of developing reliable, secure, resilient software. I just don't think you gain a lot of reliability, security or resilience by mandating a concrete&steel OS foundation. For sure, a shoddy, insecure, duck-tape application on some homebrew Linux can be ported RedHat just fine. There is nothing preventing insecure use of "Military Grade" AES256 encryption.
You just can't certify your way to resilient software.
No, but this will largely target producers of software. So they'll have to show they've done their due diligence by probably producing documentation about procedures they followed, designs, conformance to the designs, procedures for on-boarding dependencies, etc. etc. That's no their real shield, however. That's the insurance policy they buy from Lloyd's, who will require that they check all the boxes. They way I won't get sued, as a developer, will probably be to work for a big company that can afford all that or add clauses to my license stating it's either 'for demonstration purposes only' or 'no licensed for use under ...'. Kind of how a lot of financial information is doled out with the disclaimer it's [sic] only for entertainment purposes.
You’re ignoring the safer futures that were choked off by allowing developers to totally ignore safety and security.
People are smart, they would have figured out really solid mechanisms that gave you great software that’s actually safe. Look at any of the fields that deserve to use the word engineering for examples.
Could high risk be the ability to monitor heart rate, or control motors? Could it be example code for those abilities that make them liable?
If RaspberryPi or Alphabet releases an educational board for experimenters but has features that could be used in a way that is considered high risk and if they are considered market powers, should they keep it from interested people because they can't fully disclaim liability?
How much should they reduce their feature set so that it doesn't tangentially run into this law. Can they just disable that part of the code, or would they run into the same problem as the GTA Hot Coffee case if it isn't gutted enough.
While it has good intentions, it will likely backfire some way as the lists of prohibited things grow or the market power clauses are reinterpreted to be applied to a wider range and smaller targets.
The Administration will work with Congress and the private sector to develop legislation establishing liability for software products and services. Any such legislation should prevent manufacturers and software publishers with market power from fully disclaiming liability by contract, and establish higher standards of care for software in specific high-risk scenarios.
It's interesting that they use both the phrases "manufacturers" and "software publishers".
Obviously Microsoft, Oracle, etc are both manufacturers and software publishers but what about that random open source project? At minimum, they're "software publishers".
The "with market power" clause might seem like a protection but it's trivially easy to see many major open source packages with huge market share as having "market power". In that case, it's less about revenue and more about adoption and interop.
Frankly, this is the nightmare scenario that I thought Microsoft, et al would use to kill Open Source ~20 years ago. If the Feds demand liability, that kills OSS both from a “no company” standpoint and from a licensing “as is” standpoint.
Even if they don't outright BAN it, they could just make it painful to get approved, which is nearly the same result but lets everyone keep their hands clean.
An easy cut-off point to avoid axing Free Software is to limit to commercial activity: you want to sell software for money? You are liable. Your software depends on some library? Then you’re liable for this library too… unless you can have someone else accept liability, and trace that in the SBOM.
Concrete example: I wrote a cryptographic library, which I give away for free. No guarantee, no liability, if I make a mistake the only thing you can take away is my reputation. But what if you want to use my library, and don’t want to be liable for it? There are two alternatives: you can approach me and pay me for me to accept liability… or you can approach a reputable audit company, and have them accept liability for a given version.
Legislate like that, and I’ll still be able to contribute potentially critical code for free.
Contributing critical code for free is a non-goal, and having critical software that the creator has no liability for is an antipattern. You’re crowding out safer software with a free option and everyone is worse off for it.
And yes it is a risk. And yes I had one critical bug in 6 years, the horror. I did it anyway because it was pushing the Pareto envelope (speed/size mostly). And of course I knew what I was getting into, which is why it is not a weekend project, but a professional grade library. More explanations on my choices here: https://monocypher.org/why/
And by the way, Monocypher is currently being used to operate the TKey from Tillitis. https://tillitis.se/products/tkey/ Good luck porting libsodium there.
---
You do have a point however: actively distributed critical software with no one being liable is an anti-pattern. Though strictly speaking, my code is only critical when it starts being used, so… we’re going back to the place of amateur work: if you want to use my code, you can pay me to get a license that does not waive my liability, or you can pay someone else to audit my library, or you can be liable yourself. And if I sell my code, I should indeed be automatically liable.
The point with my example is that with proper legislation, even cryptographic libraries can continue being open source.
I don’t see how what code it is matters, the principle doesn’t change based on the people involved. The people who wrote OpenSSL thought they knew what they were doing. As did the people who approved Dual_EC_DRBG.
“Professional grade library” is an interesting choice of words in a thread about how we need to have liability for software, especially for software that is today in use exclusively licensed under no-liability licensing. If anything, all of the words on your site make it worse. A reasonable person, upon reading those fancy words, would be lead to the conclusion that this is Serious Software and thus safe to use.
Are you liable under the license they’re using? Their site has a lot of “trust me bro” security words on it, which is a great illustration of my point. You wrote it, you chose of your own free will to make it available, you’re responsible if it gets people hurt.
> The people who wrote OpenSSL thought they knew what they were doing.
The people who wrote OpenSSL had a much harder job, and my epistemic position is a lot better: I have over 20 years worth of additional collective wisdom to draw from, the primitives I implement are much* easier to deal with, and the problem I set out to solve is at least one order of magnitude easier.
> Are you liable under the license they’re using?
I’m not. And to be honest I’m on the fence on whether I should be. Nobody paid me for this work after all. On the other hand, if a law made me liable I wouldn’t complain too much to be honest.
If it’s regulation more than liability, I already followed most, if not all, the procedures outlined in the upcoming EU law on software resilience. I should have little problem going all the way to full compliance there (except perhaps if they ask for third party audits, I’m not paying 6.000€ per release).
> you’re responsible if it gets people hurt.
Believe me, I know. Thankfully so far nobody reported being hurt in any way — even from the sevcrit. And I continue making sure it stays that way.
---
By the way, I’m planning to work on an FPGA RISC-V core soon. I’m pretty sure nobody will give me any flak for this. And yet when you look at it, this is even more critical than a cryptographic library.
> The best I can do here is "trust me bro", and a bit of social validation.
You seem to have totally missed the point of this thread. That’s not the best you can do, it’s merely the best you’re willing to do. You’re so convinced that the internet should trust your code? Put that into the license. You have the directionality wrong; nobody asked you to release dangerous code while disclaiming all liability. You decided to do that.
I dealt with closed source enterprise web servers back in the 90s. I am pretty sure Apache and nginx are safer. Maybe there are examples where that's not true, but my gut feeling is that "all eyes are shallow". Or something.
Httpd was an absolute disaster from a security standpoint in the 90s, as was everything. Modsecurity didn’t even come out until 2002. Comparing nginx (released in 2004) is apples and oranges to say the least.
OpenSSL or log4j would like a word about how shallow bugs are in popular software.
The "private sector" here will be FAANG and other huge tech cos. They will argue for massive liability. Congress will agree because it makes them look tough and decisive. They know that only they can afford such liability. Smaller companies will be crushed by the liability insurance they need to cover these requirements.
IMO, every software dev using node, with endless thousands of packages, all with unknown licenses, and with unknown origins, unaudited, and by default, the ability to execute scripts and code during install...
And which if you audit all those packages, 10 seconds later they could be sold/stolen/hijacked for malicious intent...
Yet devs think this is awesome...
Shows why we're stuck gping dpwn this path.
Because devs have shown, they do not care about security, at all.
That's it. Devs are cobbling together systems as quickly as possible from what are effectively black boxes. There's minimal insight into their underlying behaviour.
> devs have shown, they do not care about security, at all.
Could add a grain of salt there, but I get your sentiment and would not want to argue their are big problems in the software industry, supply chain being one.
We could be angry about that, but what can be done to improve the situation? Specifically: what should the government do?
Proposed solution of adding a safe harbor framework to shield [companies] from liability is like saying: don't use npm, use our blessed supply chain. Sounds like a bold experiment, but IMO it's not obvious that this will curtail the reckless behaviours of those software devs you refer to.
Is the blessing done by you, someone in your team, someone in another team, someone in another company, someone in government?
Is the blessed supply chain comparable (not in volume, but in features, ease of use) to NPM or pip ecosystem? What do you do when you need the feature that isn't in the blessed set?
It's done by another team. It's use is generally comparable to NPM and pip. In fact we use pip, but we pull from our servers instead of public.
A lot of the scanning is automated and takes care of problems I've actually run into, like virus laden files. if something is not available, we don't use it. Because the cost of violating that policy is worse than having to find another solution.
One useful side benefit of using a private package repo is telemetry. If every dev and CI/CD machine comes with your private pypi, and you want to know where some bad version was used, you can just check the logs as opposed to playing scanner roulette. Even if you did no scanning and literally just proxies public pypi, this would make it worth it.
> IMO, every software dev using node, with endless thousands of packages, all with unknown licenses, and with unknown origins, unaudited, and by default, the ability to execute scripts and code during install...
And which if you audit all those packages, 10 seconds later they could be sold/stolen/hijacked for malicious intent...
Yet devs think this is awesome...
Like most devs (or engineering managers), I’m resigned to modern realities.
It’s a race to the bottom and I’ve been dragged into flame wars by smug (and very wrong) engineers who believed semantic versioning would solve compatibility problems and the like.
It’s impossible to justify the cost of full auditing and ground-up building. The only way to solve this is to force ground rules.
It’s really easy to develop (especially backend) code that doesn’t have this horrible problem. Just use literally any other language. This is neither modern nor a reality.
Regulation can’t come to this snake oil industry fast enough.
Another way to see this is that regulation can drastically change "modern reality". Require a software bill of materials and the NPM dependency nightmare will soon cease to be a reality.
I’ve heard this before, but I don’t understand why people think this. Having an SBOM for a lovecraftian horror won’t make it less of a horror, and it will give people something to point at and say “it must be safe, it’s got this cryptography stuff!”
What I’m saying is, with the SBOM you get to see the Lovecraftian horror, and it such a drag and a time sink to properly trace the transitive closure of all your dependencies that soon, you end up thinking of maybe having fewer of them.
Besides, I’m of the opinion that knowing your dependencies is a requirement for all serious software. (Meaning, if you’re not already tracking your dependencies, you are not taking your own software seriously.)
On one hand, there are a ton of downsides from the government sticking their noses into this - development will become slower, more costly, and in many cases no more secure or higher quality.
On the other hand, as a rando off the street consumer, I appreciate that an FDA exists to help make sure someone selling me bread isn’t mixing sawdust in the flour to help juice those sweet startup bakery numbers. There are plenty of places to criticize the FDA, but we wouldn’t need that regulatory oversight if the public weren’t being harmed by such practices.
Sawdust is unfortunately an actual food ingredient you’re more likely to find in industrial food regulated by the FDA. There’s an argument to be made that the free market, mom and pop/artisanal providers will be of higher quality than something regulated by the government.
As long as it’s food safe saw dust then the FDA is doing it’s job.
We should be perfectly happy to trade in higher quality but dangerous products for lower quality but safe ones. The airline industry kills many fewer people than it used to, and most crashes and fatalities happen in the less regulated parts. Because of course they do.
There's no evidence that mom and pop shops are any less likely to put borax in their 100% organic, gluten free, artisanal milk than a large company. Or even just washing the kitchen you can't really see. It's food safety laws and regulation that also keep them in line.
If you’re interested, The Poison Squad by Deborah Blum is both horrifying and fascinating read about the food industry in late 1800s - early 1900s, and the foundation of the FDA.
At the time, it was common to put formaldehyde in meat and dairy, borax in butter, salicylic acid into wine, and to sell products such as “pepper dust”, which was literal dust with no real pepper in it.
Seeing the regulation and accountability hammers falling upon us sure is scary.
But.
Isn’t it past time we actually get accountable for our work? Isn’t it past time we provide some form of guarantee of merchantability and suitability for some purpose? Sure amateur work should not be forbidden. But come on, when we sell something for profit, we should be accountable when it falls short of its promises.
I don’t know how to best handle it. I’m even pretty sure some unintended consequences are inevitable (though they can probably be minimised). Still, do we really want to keep the status quo, where we can sell insecure software and never be liable for this? Most other industries are liable already, why wouldn’t we?
One thing’s for sure: voting with our wallets clearly isn’t enough. One way or another our market needs a little reshaping.
Most tech is heavily dependent on other tech, and networks are often only best effort. So I'd agree industrial, auto, and medical software should have some more regulation around safety and security; but not all commercial work.
I don't see anything particularly disqualifying raised in this take on the new cyber direction.
Its concerns around limiting free expression, duty to adhere to standards of care, and market power could be alleviated by suitable legal tests.
I agree liability for vulns in court is problematic because of the elaborate possible futures that can occur in court, and the costs involved, but I see that this would likely be set by a sensible implementation of these ideas as a "last resort", with plenty of escape hatches to avoid it on the way, leaving the effect not toothless but just.
I'm interested to see where this will go, I don't think it will be a bad thing for my company's cybersecurity products.
A potential for abuse tho I think is the misuse of whatever collection of laws and rules results, to compel and gag deliberate weakening for the purposes of maintaining backdoor access, which should instead be negotiated through dedicated channels (warrants etc).
Whenever there's commentary on new laws, it's pertinent to ask what lobbyists might be representing what interests in any discussion? I don't know the answer here, but I'd like to see speculation.
One awesome thing that I hope results from this is that more lay people get somewhat conversant in what constitutes secure by design software. An increase in such security literacy can only be good--I think! Unless it's like that situation where people go to their doctors with their diagnosis and treatment already worked out thanks to Google, and are just looking for the doc to rubber stamp it! Which...depending on you and your doc, might be the right choice! Haha :)
I think I would rephrase one sentence 'what constitutes secure software by design.' to 'it may make some places actually realize there's an activity called software design, which is when you think about how the software should be have before you whack something together.'
> more lay people get somewhat conversant in what constitutes secure by design software
This will never happen anymore than laypeople will start thinking about the engineering principles behind building the reliable internal combustion engine in our cars. We want things that work without us thinking about it, and that's the end of it.
Hmm, I don't know about that. While I agree with your point that we shouldn't have to think about the intricacies of operation of nice things, to some extent that's not true and there are things we should think about, because we need to be responsible ourselves.
In other words, I think there's a certain level of trickle down expertise, and in fact, verily that's how culture grows its knowledge over time. Look at the history of hygeine: at first people laughed in the face of invisible germs, now people know to wash hands, and even, recently, they wear masks when sick. Among a plethora of other hygenic practices. Yes, there will always be hold outs: dubious food-truck kitchens I be lookin' at ya--but overall I think people do learnings over time and pass em down to the subsequent generationisms. Or don't ya think so? :) ;p xx
How is this different from let’s say Banks, having to be audited by a third party and said reports reviewed by a Banking Regulator that can fine or close said Bank?
And if controls fail and a Bank buckles down, they need to respond with their own capital. (Political favors and bailouts aside)
That's a fair point. I think the casualties will be the stuff that's released that's not directly monetized. The open source developer doesn't make any money to justify paying for any of the process to get a seal of approval. Nor would they generally want to spend their evening preparing paperwork for someone else to review. A lot of critical projects have 'teams' that are basically 1 person.
And what's considered 'critical' might be overly broad. Is the manufacture of Kleenex brand facial tissues critical? It might be under the final law. If your SaaS is used by Kleenex, do you now have to produce design documents, software process documents, audits of your process, etc. to show you can continue to be used by Kleenex to produce a trusted name brand in facial tissues? Do you do all the work to keep Kleenex as a customer only to have them switch to your Indian competitor who does none of that?
Regulation in general is way past overdue for software.
How many other world changing technologies were allowed to become so utterly fundamental to the functioning of society before we started making rules to set a minimum standard?
This has actually been a common pattern in the industrial revolution:
1. Invent new technology that revolutionizes society (e.g. railroad, telephone, software)
2. Quickly amass a fortune exploiting this new technology.
3. Use a portion of that fortune to lobby for industry regulation under the guise of concern for customers which just so happens to also raise the ladder and make it harder for startups to disrupt the incumbents, thus securing their fortune.
That said, even with the cynical take, I find it hard not to be in favor of regulation (assuming it's done well, which has been a bit hit and miss over the years). A lot of software has potentially severe liabilities and it's time companies took responsibility for those instead of shifting them onto customers.
It does create moats where non existed before. I do think the government over-estimates how well and up to date it keeps its standards. I don't know if it's still true, but 8 years ago (2015), you had to submit documentation to the FDA on CDROM. No. You couldn't just upload it.
1. The most obvious direction is that everything slows down in development. Innovation is killed in established businesses, startups need more initial investment.
2. If startups are not affected like in the EU, then there will be an explosion of them as large companies outsource their innovation or spin off their own startups. Innovation within large companies will still suffer. Maybe that's a good thing.
3. Greater focus is put into fewer projects to develop a stronger ecosystem and engineering culture so that we can increase productivity and still have time for strong standards. Might end up like 1 anyways.
Sometimes slowing down and better process and standards are two parts of the same coin. While unit testing might speed you up on net, creating design specs, design reviews, architecture reviews, control boards for configuration changes, etc. slow things down. Sometimes in a useless way and sometimes in a good way. It all depends.
But I doubt the EU would be any more lax over time.
4. Nowhere, after spending an enormous amount of money.
And it could spawn a boom for compliance consultancies that can maneuver the regulations and check the boxes on behalf of companies so that they can remain free from liability. All without any meaningful impact.
Unfortunately, I concur that this is the most likely outcome.
I would love to believe we could inch a little toward:
> 3. Greater focus is put into fewer projects to develop a stronger ecosystem and engineering culture [...]
A hopeful sign might be stuff like PCI compliance: major impact on velocity, money wasted on consultants, brain-dead training and project managers, but also much less catastrophes than what might have been. But it's just as likely it goes the SLA-way, where promises and legal documents paint some desirable level of control that in reality does not exist, just because asses need covering.
> large companies outsource their innovation or spin off their own startups
Large companies will love this because now they can pay their engineers with options on the "startup" which will most likely fail rather than RSUs on the main company which are as good as cash.
What we have now is kind of untenable, but I'd argue it's not about the incentives for developers, it's what customers will pay for. I had one customer, as [sic] government agency, lecture me about how sensitive their documents were and had to be protected. They even displayed them as images, not as text, to prevent 'copy and paste.' (I'll wait until you're done laughing).
I then mentioned that in their current software, all the URLs were directly accessible, since the image requests were not checked by their terrible, terrible, hand-made, bespoke, authentication and authorization system. They said basically that was okay, because the URLs were complicated. They weren't, it was a path and a sequential image id. They wanted to pay for features but not to fix an obvious security hole. And that's probably the norm for most customers - why do they want to spend 300k fixing bad, old code that they think works?
Most developers do care and try to make correct software. Some developers are ass-hats that are barely competent, going from project to project before they are 'found out.' Some contractors will put anyone with a pulse on the contract to avoid losing billable hours. And when's the last time you heard someone putting off new features if their competitors were already rolling similar features out?
That being said, I can see this resulting in the market for at least some products completely drying up, or getting very expensive. It would basically cement some players into the market, like IBM/RedHat for Linux, ARM and their toolchains for embedded devices, or Microsoft/AWS for cloud. It would be a way to create a moat against competition. It will probably spawn and insurance market, making it impossible for small developers to also purchase the policy to release their software. Maybe you could add additional clauses to open source licenses, such as 'not licensed for use in any sensitive or critical system as defined by USC x.y.z'.
And it still doesn't fix the problem of the pointy-haired boss ignoring risks or releasing too early to meet a schedule or competitive pressure.
One thing I have observed when giving security advice (and having the street cred to do so): people don’t believe it. You tell them no, really 2+2 is not 5, here’s the Peano arithmetic to prove it (2+2 = 1+3 = 0+4 = 4), and they don’t believe you.
Real example, I was working on TPM provisioning once. There was something fishy about the process: though they ostensibly wanted a secure channel between the provider and the TPM, the connection procedure didn’t properly authenticate the TPM. It could be replaced by a fake or emulated TPM, the provider would be none the wiser. Not too big a deal (it all happens inside the factory), but still worth pointing out. Anyway, my team lead did not believe me.
Fast forward a number of weeks (the TPM is a freaking mess, I was slow), most of the provisioning was done, secure channel and all, the whole procedure worked without a hitch… on an emulated TPM. Told you so: we need to validate the public key shown in the certificate the TPM provides, else it could come from any manufacturer. Anyway, I had working code to prove my case, and my team lead still did not believe me.
Me, the guy who wrote a cryptographic library, and was put to work on the TPM precisely because of that expertise. I had to escalate to the security guy on a meeting, and he believe me. Probably because he actually knew his shit. So at last, I was allowed to fix the bug.
The lesson I draw from that is that demonstrating a security exploit is freaking important. It’s not enough to point out the bridge isn’t finished, people need to see a car falling down to its doom.
While there is an increasing amount of noise from various sectors about what not to do we as an industry need to be concerned with what to do.
I would have a lot more faith in regulation as a spur for security innovation if the overwhelming proportion of self labeled security people weren’t just obstructionist grifters.
And I would have a lot more faith in software developers if they didn’t go all in with npm madness and like to tell anyone that points this out that they’re out of touch.
Or when we collectively take a decentralized system and use it in a centralized way because we are too lazy.
One of the clearest things we must get a handle on is auditing where code came from, and on which machines, under which jurisdictions, any building, packaging, signing etc. occurred.
I would tend to also believe we need to have much stronger code permissions enforcement, especially moving towards a whitelist only model around network host access. If we can enable only sub sections of our code to have such permissions so much the better.
What we must not do is allow a few government supported gatekeeping organizations to act as man in the middle signatories, where dealing with them is mandatory to deploying anything. That is a government run App Store.
I always lobby actively for golang because of this. Golang is a "batteries included" language ecosystem, where golang.org/x is the package that implements almost all RFCs you could ever think of.
In most industries that I worked in, management folks do not understand how graciously they have fucked up for politicians to care about something unmeasurable as cyber security.
A lot of developers I have spoken with think that workflow patterns like Test Driven or Domain Driven Design will make their code inherently more secure. Even when you point out absolute rookie mistakes like missing sanitization of user input they're always smart-assing you about how this will never affect anything that they built. If you then point out RCEs in libraries they keep reintroducing for whatever opinionated reasons they deflect the blame upon the random guy on the internet that built this library on the weekend.
I am glad the US made the first move after the pipeline incident, and I am glad that government projects started with SBOMs. Why? Because it showed up how little companies know about their risk management of operational dependencies.
It's easy to blame the log4j maintainer, but it isn't easy to blame the guy for a mistake that he made more than 20 years ago, and fixed it meanwhile while already documenting it. Because that is totally upon software/development managers and the total lack of quality assurances and testing in their workflows.
And that is talking only about something as easily fixable as a java dependency in your xml file.
Imagine how messed up security could be if there's a babel library affected, and joyent would not hard-fork it to force-fix it for everyone. There's no way to guarantee anything in npm, by nature of the pre/post hooks in its package manifest format.
> think that workflow patterns like Test Driven or Domain Driven Design will make their code inherently more secure
But this is exactly the type of mistake that government bureaucrats always make. They focus completely on procedure. There will be a whole bunch of boxes to check, including "this project uses TDD," "this project uses Domain Driven Design." Check them, and you are declared "secure." This is why such regulation will fail to provide any security, and will just slow down development.
And the max government salary is around $170K, no bonus, no equity. It's not like they're going to be able to hire any competent engineers to write these rules. They even had to build the US Digital Service, which is basically a charity case (for the richest most powerful org on earth, lol) to even get their house partially in order.
I welcome this change, greatly. The world of software engineering is an awful shitshow of cowboys, vulnerabilities, amateurism and mud balls. The metaphorical well barely contains any water, it’s almost entirely piss. It’s disgusting and reduces the reputation of the work good engineers can actually achieve. More regulation is a good thing to weed out the amateurs from the professionals and to bring some value back to what we do.
The root cause of this foolishness continues to go un-noticed.
Imagine a parallel world where where we no fuses, nor circuit breakers, nor any type of thing. You'd have government very carefully regulating what could and couldn't be hooked to the local power grid, just like this.
We live in a parallel universe to that, where nobody even considers only giving a program or "app" access to a carefully contained set of resources, instead of the whole computer, and internet. We find "as if" solutions, like VMs or containers, while still ignoring the root issue.
Imagine when the clowns running Australia go for the same thing. They just blew 2.8B last week on a failed IT project. It happens once every few months. While the private industry struggles to deliver IT, the government here completely fails.
Now, they start to tell the private sector how to do things. LOL.
81 comments
[ 3.5 ms ] story [ 187 ms ] threadControl would make sense if governments had a clear, flawless track record of developing software; that state of perfection could be attributed (solely) to a bunch of existing government guidelines and the only problem would be that developers fail to meet those guidelines. We're not there.
Why? The government does not have a clear, flawless track record developing food, nor medicine, still it controls and regulates them for the safety of society.
You can't demand absolute knowledge as a pre-requisite of regulations, that's absurd and unachievable. Extremely black-and-white thinking as well.
What I meant to say is that as it stands most governments aren't a shining example when it comes to overseeing their own software efforts.
Windows, back in the day, was both buggy and full of security holes. It was also $99, and enabled an inexpensive computer to do near-workstation feats. If it cost, say, three times as much, that would have changed the market dynamic quite a bit. Fewer of us would have been able to afford computers with Windows, and few of us wanted bare-metal computers.
Well, you say, that's just fine, we could have put Linux on it, and the world would have been a better place. Well, could you have put Linux on it? Can Linux survive in this regulatory environment? Can it survive as open as it is, as modifiable as it is, as free as it is? What about the rest of free software, that doesn't necessarily have a large company behind it?
The second-order effects of this could completely change the software landscape. We may not like the results.
Given that everything's getting either a WiFi or bluetooth chip in it, could we see coffee makers that are not approved for use in a commercial, health-care, industrial, or business setting?
You just can't certify your way to resilient software.
People are smart, they would have figured out really solid mechanisms that gave you great software that’s actually safe. Look at any of the fields that deserve to use the word engineering for examples.
Could high risk be the ability to monitor heart rate, or control motors? Could it be example code for those abilities that make them liable?
If RaspberryPi or Alphabet releases an educational board for experimenters but has features that could be used in a way that is considered high risk and if they are considered market powers, should they keep it from interested people because they can't fully disclaim liability?
How much should they reduce their feature set so that it doesn't tangentially run into this law. Can they just disable that part of the code, or would they run into the same problem as the GTA Hot Coffee case if it isn't gutted enough.
While it has good intentions, it will likely backfire some way as the lists of prohibited things grow or the market power clauses are reinterpreted to be applied to a wider range and smaller targets.
It's interesting that they use both the phrases "manufacturers" and "software publishers".
Obviously Microsoft, Oracle, etc are both manufacturers and software publishers but what about that random open source project? At minimum, they're "software publishers".
The "with market power" clause might seem like a protection but it's trivially easy to see many major open source packages with huge market share as having "market power". In that case, it's less about revenue and more about adoption and interop.
Frankly, this is the nightmare scenario that I thought Microsoft, et al would use to kill Open Source ~20 years ago. If the Feds demand liability, that kills OSS both from a “no company” standpoint and from a licensing “as is” standpoint.
Even if they don't outright BAN it, they could just make it painful to get approved, which is nearly the same result but lets everyone keep their hands clean.
Concrete example: I wrote a cryptographic library, which I give away for free. No guarantee, no liability, if I make a mistake the only thing you can take away is my reputation. But what if you want to use my library, and don’t want to be liable for it? There are two alternatives: you can approach me and pay me for me to accept liability… or you can approach a reputable audit company, and have them accept liability for a given version.
Legislate like that, and I’ll still be able to contribute potentially critical code for free.
And yes it is a risk. And yes I had one critical bug in 6 years, the horror. I did it anyway because it was pushing the Pareto envelope (speed/size mostly). And of course I knew what I was getting into, which is why it is not a weekend project, but a professional grade library. More explanations on my choices here: https://monocypher.org/why/
And by the way, Monocypher is currently being used to operate the TKey from Tillitis. https://tillitis.se/products/tkey/ Good luck porting libsodium there.
---
You do have a point however: actively distributed critical software with no one being liable is an anti-pattern. Though strictly speaking, my code is only critical when it starts being used, so… we’re going back to the place of amateur work: if you want to use my code, you can pay me to get a license that does not waive my liability, or you can pay someone else to audit my library, or you can be liable yourself. And if I sell my code, I should indeed be automatically liable.
The point with my example is that with proper legislation, even cryptographic libraries can continue being open source.
“Professional grade library” is an interesting choice of words in a thread about how we need to have liability for software, especially for software that is today in use exclusively licensed under no-liability licensing. If anything, all of the words on your site make it worse. A reasonable person, upon reading those fancy words, would be lead to the conclusion that this is Serious Software and thus safe to use.
Are you liable under the license they’re using? Their site has a lot of “trust me bro” security words on it, which is a great illustration of my point. You wrote it, you chose of your own free will to make it available, you’re responsible if it gets people hurt.
> The people who wrote OpenSSL thought they knew what they were doing.
The people who wrote OpenSSL had a much harder job, and my epistemic position is a lot better: I have over 20 years worth of additional collective wisdom to draw from, the primitives I implement are much* easier to deal with, and the problem I set out to solve is at least one order of magnitude easier.
> Are you liable under the license they’re using?
I’m not. And to be honest I’m on the fence on whether I should be. Nobody paid me for this work after all. On the other hand, if a law made me liable I wouldn’t complain too much to be honest.
If it’s regulation more than liability, I already followed most, if not all, the procedures outlined in the upcoming EU law on software resilience. I should have little problem going all the way to full compliance there (except perhaps if they ask for third party audits, I’m not paying 6.000€ per release).
> you’re responsible if it gets people hurt.
Believe me, I know. Thankfully so far nobody reported being hurt in any way — even from the sevcrit. And I continue making sure it stays that way.
---
By the way, I’m planning to work on an FPGA RISC-V core soon. I’m pretty sure nobody will give me any flak for this. And yet when you look at it, this is even more critical than a cryptographic library.
You seem to have totally missed the point of this thread. That’s not the best you can do, it’s merely the best you’re willing to do. You’re so convinced that the internet should trust your code? Put that into the license. You have the directionality wrong; nobody asked you to release dangerous code while disclaiming all liability. You decided to do that.
OpenSSL or log4j would like a word about how shallow bugs are in popular software.
The "private sector" here will be FAANG and other huge tech cos. They will argue for massive liability. Congress will agree because it makes them look tough and decisive. They know that only they can afford such liability. Smaller companies will be crushed by the liability insurance they need to cover these requirements.
And which if you audit all those packages, 10 seconds later they could be sold/stolen/hijacked for malicious intent...
Yet devs think this is awesome...
Shows why we're stuck gping dpwn this path.
Because devs have shown, they do not care about security, at all.
Could add a grain of salt there, but I get your sentiment and would not want to argue their are big problems in the software industry, supply chain being one.
We could be angry about that, but what can be done to improve the situation? Specifically: what should the government do?
Proposed solution of adding a safe harbor framework to shield [companies] from liability is like saying: don't use npm, use our blessed supply chain. Sounds like a bold experiment, but IMO it's not obvious that this will curtail the reckless behaviours of those software devs you refer to.
Is the blessing done by you, someone in your team, someone in another team, someone in another company, someone in government?
Is the blessed supply chain comparable (not in volume, but in features, ease of use) to NPM or pip ecosystem? What do you do when you need the feature that isn't in the blessed set?
A lot of the scanning is automated and takes care of problems I've actually run into, like virus laden files. if something is not available, we don't use it. Because the cost of violating that policy is worse than having to find another solution.
Like most devs (or engineering managers), I’m resigned to modern realities.
It’s a race to the bottom and I’ve been dragged into flame wars by smug (and very wrong) engineers who believed semantic versioning would solve compatibility problems and the like.
It’s impossible to justify the cost of full auditing and ground-up building. The only way to solve this is to force ground rules.
Regulation can’t come to this snake oil industry fast enough.
Besides, I’m of the opinion that knowing your dependencies is a requirement for all serious software. (Meaning, if you’re not already tracking your dependencies, you are not taking your own software seriously.)
Maybe other platforms don’t have the same scale of horribleness as the web stacks, but there’s still a problem.
For the dotnet ecosystem, for instance, I’m not reinventing functionality when nuget packages exist.
But the same concerns persist because I’m not really sure if it’s secure, without auditing.
Both from a security and compatibility perspective, it's not a solved problem because it's not a problem specific to any tech stack.
On the other hand, as a rando off the street consumer, I appreciate that an FDA exists to help make sure someone selling me bread isn’t mixing sawdust in the flour to help juice those sweet startup bakery numbers. There are plenty of places to criticize the FDA, but we wouldn’t need that regulatory oversight if the public weren’t being harmed by such practices.
We should be perfectly happy to trade in higher quality but dangerous products for lower quality but safe ones. The airline industry kills many fewer people than it used to, and most crashes and fatalities happen in the less regulated parts. Because of course they do.
tiny, very tiny, particles of saw dust (labeled "cellulose" in the ingredients), is used to keep the cheese from clumping up.
go and watch your favorite pizza chain make pizza, with giant bags of pre-shredded cheese, and then ponder your sawdust intake.
Edit: while we’re playing food trivia though, ponder the no-sawdust rule that doesn’t forbid cinnamon.
At the time, it was common to put formaldehyde in meat and dairy, borax in butter, salicylic acid into wine, and to sell products such as “pepper dust”, which was literal dust with no real pepper in it.
But.
Isn’t it past time we actually get accountable for our work? Isn’t it past time we provide some form of guarantee of merchantability and suitability for some purpose? Sure amateur work should not be forbidden. But come on, when we sell something for profit, we should be accountable when it falls short of its promises.
I don’t know how to best handle it. I’m even pretty sure some unintended consequences are inevitable (though they can probably be minimised). Still, do we really want to keep the status quo, where we can sell insecure software and never be liable for this? Most other industries are liable already, why wouldn’t we?
One thing’s for sure: voting with our wallets clearly isn’t enough. One way or another our market needs a little reshaping.
Single player game: maybe we want to cut back on the skinner boxes, or automatically mark 18+ any actual gambling.
Professional accounting: maybe we want the thing to be error-free under normal operating conditions (good luck defining "normal" here though).
Media player: that thing routinely reads stuff from the Internet, maybe it should be robust to hostile input.
This is true in almost every industry, especially medicine. There is no accountability.
It's a myth like the fairy godmother, etc.
Its concerns around limiting free expression, duty to adhere to standards of care, and market power could be alleviated by suitable legal tests.
I agree liability for vulns in court is problematic because of the elaborate possible futures that can occur in court, and the costs involved, but I see that this would likely be set by a sensible implementation of these ideas as a "last resort", with plenty of escape hatches to avoid it on the way, leaving the effect not toothless but just.
I'm interested to see where this will go, I don't think it will be a bad thing for my company's cybersecurity products.
A potential for abuse tho I think is the misuse of whatever collection of laws and rules results, to compel and gag deliberate weakening for the purposes of maintaining backdoor access, which should instead be negotiated through dedicated channels (warrants etc).
Whenever there's commentary on new laws, it's pertinent to ask what lobbyists might be representing what interests in any discussion? I don't know the answer here, but I'd like to see speculation.
One awesome thing that I hope results from this is that more lay people get somewhat conversant in what constitutes secure by design software. An increase in such security literacy can only be good--I think! Unless it's like that situation where people go to their doctors with their diagnosis and treatment already worked out thanks to Google, and are just looking for the doc to rubber stamp it! Which...depending on you and your doc, might be the right choice! Haha :)
This will never happen anymore than laypeople will start thinking about the engineering principles behind building the reliable internal combustion engine in our cars. We want things that work without us thinking about it, and that's the end of it.
In other words, I think there's a certain level of trickle down expertise, and in fact, verily that's how culture grows its knowledge over time. Look at the history of hygeine: at first people laughed in the face of invisible germs, now people know to wash hands, and even, recently, they wear masks when sick. Among a plethora of other hygenic practices. Yes, there will always be hold outs: dubious food-truck kitchens I be lookin' at ya--but overall I think people do learnings over time and pass em down to the subsequent generationisms. Or don't ya think so? :) ;p xx
And if controls fail and a Bank buckles down, they need to respond with their own capital. (Political favors and bailouts aside)
And what's considered 'critical' might be overly broad. Is the manufacture of Kleenex brand facial tissues critical? It might be under the final law. If your SaaS is used by Kleenex, do you now have to produce design documents, software process documents, audits of your process, etc. to show you can continue to be used by Kleenex to produce a trusted name brand in facial tissues? Do you do all the work to keep Kleenex as a customer only to have them switch to your Indian competitor who does none of that?
Inquiring minds want to know.
How many other world changing technologies were allowed to become so utterly fundamental to the functioning of society before we started making rules to set a minimum standard?
1. Invent new technology that revolutionizes society (e.g. railroad, telephone, software) 2. Quickly amass a fortune exploiting this new technology. 3. Use a portion of that fortune to lobby for industry regulation under the guise of concern for customers which just so happens to also raise the ladder and make it harder for startups to disrupt the incumbents, thus securing their fortune.
That said, even with the cynical take, I find it hard not to be in favor of regulation (assuming it's done well, which has been a bit hit and miss over the years). A lot of software has potentially severe liabilities and it's time companies took responsibility for those instead of shifting them onto customers.
1. The most obvious direction is that everything slows down in development. Innovation is killed in established businesses, startups need more initial investment.
2. If startups are not affected like in the EU, then there will be an explosion of them as large companies outsource their innovation or spin off their own startups. Innovation within large companies will still suffer. Maybe that's a good thing.
3. Greater focus is put into fewer projects to develop a stronger ecosystem and engineering culture so that we can increase productivity and still have time for strong standards. Might end up like 1 anyways.
But I doubt the EU would be any more lax over time.
And it could spawn a boom for compliance consultancies that can maneuver the regulations and check the boxes on behalf of companies so that they can remain free from liability. All without any meaningful impact.
I would love to believe we could inch a little toward:
> 3. Greater focus is put into fewer projects to develop a stronger ecosystem and engineering culture [...]
A hopeful sign might be stuff like PCI compliance: major impact on velocity, money wasted on consultants, brain-dead training and project managers, but also much less catastrophes than what might have been. But it's just as likely it goes the SLA-way, where promises and legal documents paint some desirable level of control that in reality does not exist, just because asses need covering.
Large companies will love this because now they can pay their engineers with options on the "startup" which will most likely fail rather than RSUs on the main company which are as good as cash.
I then mentioned that in their current software, all the URLs were directly accessible, since the image requests were not checked by their terrible, terrible, hand-made, bespoke, authentication and authorization system. They said basically that was okay, because the URLs were complicated. They weren't, it was a path and a sequential image id. They wanted to pay for features but not to fix an obvious security hole. And that's probably the norm for most customers - why do they want to spend 300k fixing bad, old code that they think works?
Most developers do care and try to make correct software. Some developers are ass-hats that are barely competent, going from project to project before they are 'found out.' Some contractors will put anyone with a pulse on the contract to avoid losing billable hours. And when's the last time you heard someone putting off new features if their competitors were already rolling similar features out?
That being said, I can see this resulting in the market for at least some products completely drying up, or getting very expensive. It would basically cement some players into the market, like IBM/RedHat for Linux, ARM and their toolchains for embedded devices, or Microsoft/AWS for cloud. It would be a way to create a moat against competition. It will probably spawn and insurance market, making it impossible for small developers to also purchase the policy to release their software. Maybe you could add additional clauses to open source licenses, such as 'not licensed for use in any sensitive or critical system as defined by USC x.y.z'.
And it still doesn't fix the problem of the pointy-haired boss ignoring risks or releasing too early to meet a schedule or competitive pressure.
Have you considered hidden watermarks just like printers? :)
Real example, I was working on TPM provisioning once. There was something fishy about the process: though they ostensibly wanted a secure channel between the provider and the TPM, the connection procedure didn’t properly authenticate the TPM. It could be replaced by a fake or emulated TPM, the provider would be none the wiser. Not too big a deal (it all happens inside the factory), but still worth pointing out. Anyway, my team lead did not believe me.
Fast forward a number of weeks (the TPM is a freaking mess, I was slow), most of the provisioning was done, secure channel and all, the whole procedure worked without a hitch… on an emulated TPM. Told you so: we need to validate the public key shown in the certificate the TPM provides, else it could come from any manufacturer. Anyway, I had working code to prove my case, and my team lead still did not believe me.
Me, the guy who wrote a cryptographic library, and was put to work on the TPM precisely because of that expertise. I had to escalate to the security guy on a meeting, and he believe me. Probably because he actually knew his shit. So at last, I was allowed to fix the bug.
The lesson I draw from that is that demonstrating a security exploit is freaking important. It’s not enough to point out the bridge isn’t finished, people need to see a car falling down to its doom.
I would have a lot more faith in regulation as a spur for security innovation if the overwhelming proportion of self labeled security people weren’t just obstructionist grifters.
And I would have a lot more faith in software developers if they didn’t go all in with npm madness and like to tell anyone that points this out that they’re out of touch.
Or when we collectively take a decentralized system and use it in a centralized way because we are too lazy.
One of the clearest things we must get a handle on is auditing where code came from, and on which machines, under which jurisdictions, any building, packaging, signing etc. occurred.
I would tend to also believe we need to have much stronger code permissions enforcement, especially moving towards a whitelist only model around network host access. If we can enable only sub sections of our code to have such permissions so much the better.
What we must not do is allow a few government supported gatekeeping organizations to act as man in the middle signatories, where dealing with them is mandatory to deploying anything. That is a government run App Store.
In most industries that I worked in, management folks do not understand how graciously they have fucked up for politicians to care about something unmeasurable as cyber security.
A lot of developers I have spoken with think that workflow patterns like Test Driven or Domain Driven Design will make their code inherently more secure. Even when you point out absolute rookie mistakes like missing sanitization of user input they're always smart-assing you about how this will never affect anything that they built. If you then point out RCEs in libraries they keep reintroducing for whatever opinionated reasons they deflect the blame upon the random guy on the internet that built this library on the weekend.
I am glad the US made the first move after the pipeline incident, and I am glad that government projects started with SBOMs. Why? Because it showed up how little companies know about their risk management of operational dependencies.
It's easy to blame the log4j maintainer, but it isn't easy to blame the guy for a mistake that he made more than 20 years ago, and fixed it meanwhile while already documenting it. Because that is totally upon software/development managers and the total lack of quality assurances and testing in their workflows.
And that is talking only about something as easily fixable as a java dependency in your xml file.
Imagine how messed up security could be if there's a babel library affected, and joyent would not hard-fork it to force-fix it for everyone. There's no way to guarantee anything in npm, by nature of the pre/post hooks in its package manifest format.
But this is exactly the type of mistake that government bureaucrats always make. They focus completely on procedure. There will be a whole bunch of boxes to check, including "this project uses TDD," "this project uses Domain Driven Design." Check them, and you are declared "secure." This is why such regulation will fail to provide any security, and will just slow down development.
And the max government salary is around $170K, no bonus, no equity. It's not like they're going to be able to hire any competent engineers to write these rules. They even had to build the US Digital Service, which is basically a charity case (for the richest most powerful org on earth, lol) to even get their house partially in order.
Imagine a parallel world where where we no fuses, nor circuit breakers, nor any type of thing. You'd have government very carefully regulating what could and couldn't be hooked to the local power grid, just like this.
We live in a parallel universe to that, where nobody even considers only giving a program or "app" access to a carefully contained set of resources, instead of the whole computer, and internet. We find "as if" solutions, like VMs or containers, while still ignoring the root issue.
Wake up!
Now, they start to tell the private sector how to do things. LOL.