Yeah, I don't buy it. If they don't want these security reports, ignore them and continue your path. Blaming AI is just an excuse to close source. If you don't want AI to learn from your code, too late. Add genetic algorithms and fuzzing into AI and it can iterate and learn a billion times faster, no need to learn for humans.
I think people are finding ways to either enable “pro” features and at least find the right extension points to implement them easily with LLMs. Security is window dressing.
Security through obscurity is only problematic if that is the only, or a primary, layer of defense. As an incremental layer of deterrence or delay, it is an absolutely valid tactic. (Note, not commenting on whether that is the rationale here.)
They probably lack a sufficient density of people who remember why "security through obscurity" become an infamous concept. It belongs to that family of bad ideas that's superficially appealing, especially if you're still at that stage of your career at which you think past generations were full of idiots and you, alone, have discovered how to do real software development.
This seems kind of crazy. If LLMs are so stunningly good at finding vulnerabilities in code, then shouldn't the solution be to run an LLM against your code after you commit, and before you release it? Then you basically have pentesting harnesses all to yourself before going public. If an LLM can't find any flaws, then you are good to release that code.
A few years ago, I invoked Linus's Law in a classroom, and I was roundly debunked. Isn't it a shame that it's basically been fulfilled now with LLMs?
It's entirely possible to address all the LLM-found issues and get an "all green" response, and have an attacker still find issues that your LLM did not. Either they used a different model, a different prompt, or spent more money than you did.
It's not a symmetric game, either. On defense, you have to get lucky every time - the attacker only has to get lucky once.
I mean, you should definitely have _some_ level of audit by LLMs before you ship, as part of the general PR process.
But you might need thousands of sessions to uncover some vulnerabilities, and you don’t want to stop shipping changes because the security checks are taking hours to run
LLMs really are stunningly good at finding vulnerabilities in code, which is why, with closed-source code, you can and probably will use them to make your code as secure as possible.
But you won't keep the doors open for others to use them against it.
So it is, unfortunately, understandable in a way...
LLM like humans can find vulnerabilities in black boxes. We already established 30 years ago that open source is usually more secure than closed source and that security by obscurity doesn't work.
Attackers only need LLMs to be good at randomly finding one vulnerability, whereas service providers need them to be good at finding all such vulnerabilities.
After a release, attackers have effectively infinite time to throw an LLM against every line of your code - an LLM that only gets smarter and cheaper to run as time passes. In order to feel secure you’d need to do all the work you’d imagine an attacker would ever do, for every single release you ship.
Every change would introduce the possibility of a vulnerability being added to the system and one would need to run the LLM scan across the entire code base. It gets very costly in a environment where you are doing regular commits. Companies like Github already provide scanning tools for static analysis and the cost is already high for them.
As LLMs improve and adoption grows, maintaining a FOSS project is becoming more complex and more expensive in terms of time and manpower. That part is easy to understand.
It is also become a trend that LLM-assisted users are generating more low-quality issues, dubious security reports, and noisy PRs, to the point where keeping the whole stack open source no longer feels worth it. Even if the real reason is monetization rather than security, I can still understand the decision.
I suspect we will see more of this from commercial products built around a FOSS core. The other failure mode is that maintainers stop treating security disclosures as something special and just handle them like ordinary bugs, as with libxml2. In that sense, Chromium moving toward a Rust-based XML library is also an interesting development.
Open-source supporters don't have a sustainable answer to the fact that AI models can easily find N-day vulnerabilities extremely quickly and swamp maintainers with issues and bug-reports left hanging for days.
Unfortunately, this is where it is going and the open-source software supporters did not for-see the downsides of open source maintenance in the age of AI especially for businesses with "open-core" products.
Might as well close-source them to slow the attackers (with LLMs) down. Even SQLite has closed-sourced their tests which is another good idea.
The tools are available to everyone. It's becoming easier for hackers to attack you at the same speed that it's becoming easier for you to harden your systems. When everyone gains the same advantage at the same time, nothing has really changed.
It makes me think of how great chess engines have affected competitive chess over the last few years. Sure, the ceiling for Elo ratings at the top levels has gone up, but it's still a fair game because everyone has access to the new tools. High-level players aren't necessarily spending more time on prep than they were before; they're just getting more value out of the hours they do spend.
Since security exploits can now be found by spending tokens, open source is MORE valuable because open source libraries can share that auditing budget while closed source software has to find all the exploits themselves in private.
> If Mythos continues to find exploits so long as you keep throwing money at it, security is reduced to a brutally simple equation: to harden a system you need to spend more tokens discovering exploits than attackers will spend exploiting them.
This conclusion makes more sense to me, but maybe I'm too naive.
The media momentum of this threat really came with Mythos, which was like 2 or 3 weeks ago? That seems like a fairly short time to pivot your core principles like that. It sounds to me like they wanted to do this for other business related reasons, but now found an excuse they can sell to the public.
This seems similar to the lesson learned for cryptographic libraries where open source libraries vetted by experts become the most trusted.
Your average open source library isn’t going to get that scrutiny, though. It seems like it will result in consolidation around a few popular libraries in each category?
It also means that you need to extract enough value to cover the cost of said tokens, or reduce the economic benefit of finding exploits.
Reducing economic benefit largely comes down to reducing distribution (breadth) and reducing system privilege (depth).
One way to reduce distribution is to, raise the price.
Another is to make a worse product.
Naturally, less valuable software is not a desirable outcome. So either you reduce the cost of keeping open (by making closed), or increase the price to cover the cost of keeping open (which, again, also decreases distribution).
The economics of software are going to massively reconfigure in the coming years, open source most of all.
I suspect we'll see more 'open spec' software, with actual source generated on-demand (or near to it) by models. Then all the security and governance will happen at the model layer.
> to harden a system you need to spend more tokens discovering exploits than attackers will spend exploiting them.
That can't be right, can it? Given stable software, the relative attack surface keeps shrinking. Mythos does not produce exploits. Should be defenders advantage, token wise, no?
> to harden a system you need to spend more tokens discovering exploits than attackers will spend exploiting them
This is true until certain point, unless the requirement / contract itself has loophole which the attacker can exploit it without limit. But I don't think this is the case.
Let's say, if someone found an loophole in sort() which can cause denial-of-service. The cause would be the implementation itself, not the contract of sorting. People + AI will figure it out and fix it eventually.
This may be true long term but not short term. It also assumes that white hats will be as motivated as black hats – not true.
For projects with NO WARRANTY, the risk is minimal, so yes there are upsides.
For a commercial project like cal.com, where a breach means massive liability, they don’t have the resources to risk breaches in the short term for potentially better software in the long term.
> It's been a common wisdom now for decades that open source is more secure.
This is not true.
The problem rather is that the managers of many companies don't allow their programmers to apply their knowledge about security - the programmers should rather weed out new features.
I wonder if we could find a way to donate unused tokens or even local compute resources to open-source projects we support. Especially for security auditing where it could probably be somewhat more asynchronous and disconnected than the open-source developers' personal tool choices.
I like that LLMs have basically switched to the weapons business model. Buy our LLM so that the bad guy we'll sell our LLM to doesnt destroy your code. As a bonus, we'll give you a little head start. And if you're a small company that can't afford our LLM, too bad.
> > If Mythos continues to find exploits so long as you keep throwing money at it, security is reduced to a brutally simple equation: to harden a system you need to spend more tokens discovering exploits than attackers will spend exploiting them.
But this has always been the reality of security: it's always been fundamentally an economic question about which party has stronger incentives and greater resources than the other. The increasing sophistication of AI is available to both parties equally, so I don't see how AI in itself fundamentally changes the equation.
Today, it's easy to (publicly) evaluate the ability of LLMs to find bugs in open source codebases, because you don't need to ask permission. But this doesn't actually tell us the negative statement, which is that an LLM won't just as effectively find bugs in closed codebases, including through black-box testing, reverse engineering, etc.
If the null hypothesis is that LLMs are good at finding bugs, full stop, then it's unclear to me that going closed actually does much to stop your adversary (particularly as a service operator).
This is some truly exceptionally clownish attention seeking nonsense. The rationale here is complete nonsense, they just wanted to put "because AI" after announcing their completely self-serving decision. If AI cyber offense is such a concern, recognize your role as a company handling truckloads of highly sensitive information and actually fix your security culture instead of just obscuring it.
The real threat is not security but bad actors copying your code and calling it theirs.
IMHO, open source will continue to exist and it will be successful but the existence of AI is deterrent for most. Lets be honest, in recent times the only reason startups went open source first was to build a community and build organic growth engine powered by early adaptors. Now this is no longer viable and in fact it is simply helping competitors. So why do it then?
The only open source that will remain will be the real open source projects that are true to the ethos.
> Now this is no longer viable and in fact it is simply helping competitors. So why do it then?
> The only open source that will remain will be the real open source projects that are true to the ethos.
Well, the second point seems like the answer to the previous question. The original model of monetizing FOSS -- support contracts, risk indemnification, etc., for an otherwise functionally equivalent product -- will still remain viable.
But those trying to thread the needle of trying to use open-source to push a "freemium" model are now going to hit a wall: if you were withholding features from the community version in order to paywall them for the premium version, and now AI has made it easy for users to add those features back without paying you, then you're screwed. The people who were going to use AI to bypass your paywall are still not going to be your customers, but you no longer have the differentiator to put you ahead of the competitors that were already closed-source to begin with for the customers who are willing to pay.
I originally deployed Cal.com because I wanted an open-source solution. But now, why would I choose a closed-source Cal.com over Calendly? If I'm forced to go SaaS, I'll probably go with the more widely used Calendly. If I'm not forced to go SaaS, I'll forego them both, and go back to something like EasyAppointments, knowing that I won't be in conflict with the authors if I choose to add my own "premium" features to it, whether with AI or by hand. All Cal.com did here was remove any chance that I'd ever pay them anything.
I hate how this sounds...but this reads to me "we lack the confidence in our code security so we're closing the source code to conceal vulnerabilities which may exist."
I know plenty of security researchers who exclusively use Claude Code and other tools for blackbox testing against sites they don’t have the source code for. It seems like shutting down the entire product is the only safe decision here!
Risk tolerance and emotional capacity differs from one individual to another, while I may disagree with the decision I am able to respect the decision.
That said, I think it’s important to try and recognize where things are from multiple angles rather than bucket things from your filter bubble alone, fear sells and we need to stop buying into it.
We did consider arguments in both directions (e.g. easier to recreate the code, agents can understand better how it works), but I honestly think the security argument goes for open source: the OSS projects will get more scrutiny faster, which means bugs won't linger around.
Time will tell, I am in the open source camp, though.
This is the future now that AI is here. Publishing is going to be dead, look at the tea leaves, how many engineers are claiming they don’t use package managers anymore and just generate dependencies? 5 years and no one will be making an argument for open source or blogging.
Juxtapose this with the fact that many HNers will decry strong copyleft FOSS licenses as not being truly "open source" - the reality is that closed source software is still full of open-source non-copyleft dependencies. Unless you're rolling your own encryption and TCP stack, being closed source will not be the easy solution that many imagine it to be.
150 comments
[ 2.8 ms ] story [ 103 ms ] threadIt seems like an easy decision, not a difficult one.
I feel like with AI, self-hosting software reliably is becoming easier so the incentives to pay for a hosted service of an OSS project are going down.
Wanna sack a load of staff? - AI
Wanna cut your consumer products division? - AI
Wanna take away the source? - AI
A few years ago, I invoked Linus's Law in a classroom, and I was roundly debunked. Isn't it a shame that it's basically been fulfilled now with LLMs?
https://en.wikipedia.org/wiki/Linus%27s_law
It's not a symmetric game, either. On defense, you have to get lucky every time - the attacker only has to get lucky once.
But you might need thousands of sessions to uncover some vulnerabilities, and you don’t want to stop shipping changes because the security checks are taking hours to run
But you won't keep the doors open for others to use them against it.
So it is, unfortunately, understandable in a way...
It is also become a trend that LLM-assisted users are generating more low-quality issues, dubious security reports, and noisy PRs, to the point where keeping the whole stack open source no longer feels worth it. Even if the real reason is monetization rather than security, I can still understand the decision.
I suspect we will see more of this from commercial products built around a FOSS core. The other failure mode is that maintainers stop treating security disclosures as something special and just handle them like ordinary bugs, as with libxml2. In that sense, Chromium moving toward a Rust-based XML library is also an interesting development.
Great move.
Open-source supporters don't have a sustainable answer to the fact that AI models can easily find N-day vulnerabilities extremely quickly and swamp maintainers with issues and bug-reports left hanging for days.
Unfortunately, this is where it is going and the open-source software supporters did not for-see the downsides of open source maintenance in the age of AI especially for businesses with "open-core" products.
Might as well close-source them to slow the attackers (with LLMs) down. Even SQLite has closed-sourced their tests which is another good idea.
Then good, that overengineered, intentionally-crippled crap should go away.
It makes me think of how great chess engines have affected competitive chess over the last few years. Sure, the ceiling for Elo ratings at the top levels has gone up, but it's still a fair game because everyone has access to the new tools. High-level players aren't necessarily spending more time on prep than they were before; they're just getting more value out of the hours they do spend.
Since security exploits can now be found by spending tokens, open source is MORE valuable because open source libraries can share that auditing budget while closed source software has to find all the exploits themselves in private.
> If Mythos continues to find exploits so long as you keep throwing money at it, security is reduced to a brutally simple equation: to harden a system you need to spend more tokens discovering exploits than attackers will spend exploiting them.
I'd give them more credits if they use the AI slop unmaintainability argument.
The media momentum of this threat really came with Mythos, which was like 2 or 3 weeks ago? That seems like a fairly short time to pivot your core principles like that. It sounds to me like they wanted to do this for other business related reasons, but now found an excuse they can sell to the public.
(I might be very wrong here)
Your average open source library isn’t going to get that scrutiny, though. It seems like it will result in consolidation around a few popular libraries in each category?
It also means that you need to extract enough value to cover the cost of said tokens, or reduce the economic benefit of finding exploits.
Reducing economic benefit largely comes down to reducing distribution (breadth) and reducing system privilege (depth).
One way to reduce distribution is to, raise the price.
Another is to make a worse product.
Naturally, less valuable software is not a desirable outcome. So either you reduce the cost of keeping open (by making closed), or increase the price to cover the cost of keeping open (which, again, also decreases distribution).
The economics of software are going to massively reconfigure in the coming years, open source most of all.
I suspect we'll see more 'open spec' software, with actual source generated on-demand (or near to it) by models. Then all the security and governance will happen at the model layer.
That can't be right, can it? Given stable software, the relative attack surface keeps shrinking. Mythos does not produce exploits. Should be defenders advantage, token wise, no?
* a la https://news.ycombinator.com/item?id=26998308
This is true until certain point, unless the requirement / contract itself has loophole which the attacker can exploit it without limit. But I don't think this is the case.
Let's say, if someone found an loophole in sort() which can cause denial-of-service. The cause would be the implementation itself, not the contract of sorting. People + AI will figure it out and fix it eventually.
For projects with NO WARRANTY, the risk is minimal, so yes there are upsides.
For a commercial project like cal.com, where a breach means massive liability, they don’t have the resources to risk breaches in the short term for potentially better software in the long term.
This is not true.
The problem rather is that the managers of many companies don't allow their programmers to apply their knowledge about security - the programmers should rather weed out new features.
Llm's will find your issues faster, but not necessarily more accurately than a domain expert. But experts cost money and effort takes longer to apply.
Are llm's going to reduce everyone's wages because they are cheap labour?
But this has always been the reality of security: it's always been fundamentally an economic question about which party has stronger incentives and greater resources than the other. The increasing sophistication of AI is available to both parties equally, so I don't see how AI in itself fundamentally changes the equation.
If the null hypothesis is that LLMs are good at finding bugs, full stop, then it's unclear to me that going closed actually does much to stop your adversary (particularly as a service operator).
https://news.ycombinator.com/item?id=47780712
IMHO, open source will continue to exist and it will be successful but the existence of AI is deterrent for most. Lets be honest, in recent times the only reason startups went open source first was to build a community and build organic growth engine powered by early adaptors. Now this is no longer viable and in fact it is simply helping competitors. So why do it then?
The only open source that will remain will be the real open source projects that are true to the ethos.
> The only open source that will remain will be the real open source projects that are true to the ethos.
Well, the second point seems like the answer to the previous question. The original model of monetizing FOSS -- support contracts, risk indemnification, etc., for an otherwise functionally equivalent product -- will still remain viable.
But those trying to thread the needle of trying to use open-source to push a "freemium" model are now going to hit a wall: if you were withholding features from the community version in order to paywall them for the premium version, and now AI has made it easy for users to add those features back without paying you, then you're screwed. The people who were going to use AI to bypass your paywall are still not going to be your customers, but you no longer have the differentiator to put you ahead of the competitors that were already closed-source to begin with for the customers who are willing to pay.
I originally deployed Cal.com because I wanted an open-source solution. But now, why would I choose a closed-source Cal.com over Calendly? If I'm forced to go SaaS, I'll probably go with the more widely used Calendly. If I'm not forced to go SaaS, I'll forego them both, and go back to something like EasyAppointments, knowing that I won't be in conflict with the authors if I choose to add my own "premium" features to it, whether with AI or by hand. All Cal.com did here was remove any chance that I'd ever pay them anything.
This post's argument seems circular to me.
That said, I think it’s important to try and recognize where things are from multiple angles rather than bucket things from your filter bubble alone, fear sells and we need to stop buying into it.
We did consider arguments in both directions (e.g. easier to recreate the code, agents can understand better how it works), but I honestly think the security argument goes for open source: the OSS projects will get more scrutiny faster, which means bugs won't linger around.
Time will tell, I am in the open source camp, though.