I wish all the best to LiteLLM team with handling it again. I can imagine how challenging are situations like this one.
However, if anyone is looking for 20x faster and less "targeted" LiteLLM alternative - I'm work on one in Go. That's why GoModel it's called. https://github.com/ENTERPILOT/GoModel/
I myself have written a small llm proxy in Go, doing nothing but injecting the real API key to outgoing requests while itself validating virtual keys it generates, and it works great.
I have GoModel running at home, being deployed at work. There are many good reasons to run a gateway, some include
- being resilient to upstream provider downtime
- centrally managed config/controls
- visibility and allocation into usage and costs
We are deploying a gateway at work for these three anyhow. At home, GoModel's "virtual models" have been super helpful for testing out new open weights. I can change the model deployed in vLLM without having to change any of my downstream tools, which get a stable name.
IMHO defaults for uv, npm, etc should all have a cooldown for 24 - 48 hours by default, you should have to go out of your way to install something newer.
We have 7d cooldown on packages on an internal pypi proxy but that blocks security updates also, which we now have to manually exclude from the cooldown.
I’m unsure how to automate this properly, maybe an LLM is the best starting point.
They definitely wouldn't be if attackers knew that it would then bypass cooldowns!
Ultimately, you need someone to verify releases and that probably means paying for it. I know Sonatype paid solutions utilise their own internal research to allow/block new releases.
I wonder if in some cases it's not so much that people are updating too quickly, but rather, their deployment pipelines do not have pinned / frozen versions attached to their builds.
The last time I looked for information, I remember seeing that the median supply chain attack was in place for multiple months to years before being discovered. Cooldowns shorter than that seem like snake oil.
Plus some random value slow curve distribution. Basically incremental rollout from the client side. But if it's not caught at the commit time, attacker will just wait for a certain point on the curve.
No, because there are malware-finding mechanisms other than "let some unlucky guy get affected first", like automated scanners. If everyone implements a cooldown, everyone will in fact be safer.
To clarify a little further, you can select the title, you just can't tell that it's selected because the author made a "poor" (REALLY REALLY DUMB) decision to deliberately style away a basic browser feature (text highlighting).
What you don't know, it's actually a very clever trick by an LLM. By not highlighting the text, it makes you think harder and gives you actually some challenge. 99% of people want to be challenged when selecting text, so the AI decided to drop text highlighting.
Yes if you use LLM to analyse everything. Not really if you analyse diffs, with heuristics to select candidates for LLM/agent based analysis. But at the cost of missing out novel attacks.
There are over 8 billion people on Earth, and some subset of the computer systems that they all use are going to have issues, every day.
It's really not as bad as the accumulated headlines and exploitative hype make it sound.
It's similar to the mass hysteria and moral panics over things like child abduction. Media reports make certain dangers seem much more important than they really are, when considered in context.
> OSS Users
> For the time being, please pin your version to 1.106.2 or any explicit version published prior to now.
> Enterprise Customers
> There currently exists a private LiteLLM registry for enterprise customers, in which we are distributing an embargoed release that remediates a vulnerability whose disclosure date is scheduled for October 27. Our private registry is not affected. If you are an enterprise customer pulling from our private registry you are not affected.
1. there is a second vulnerability that will be disclosed next Tuesday (can't be this one because talking about it as undisclosed in the GH issue disclosing it makes no sense)
2. the patch for that vulnerability is somewhere in the diff between $current_public_release and $current_enterprise_release
3. You have to pay to get $current_enterprise_release
They might not be selling the vulnerability directly, but they are selling access to the patch that fixes it. Which would be enough of a hint even in pre-LLM times. And unlike other vendors they don't just claim "you get faster updates and additional features" or something like that, they directly point out that if you buy now you get advance access to the fix to an unannounced vuln
I'm not saying they are advertising this to bad actors. But if they wanted to do that with a shred of plausible deniability, this would be the way to do it
There exists a critical vulnerability which affects only Enterprise LiteLLM deployments, and does not affect OSS deployments.
We have provided our customers with patched releases distributed via a private registry and are allowing them some time to update their deployments before we release the fix in our public registry and disclose the vulnerability details. Releasing the fix publicly in an open source product would be a de-facto vulnerability disclosure.
I am not following how someone snuck the files into the release.
All I read was that it creates a directory from the build file but seemingly no description how new files or added dependencies slipped by. Was `cook.pth` a dependency that got automatically downloaded from PyPI on install and then the build system put it into dependencies?
--
Leaving this for others:
> BerriAI’s private release pipeline built these wheels from an attacker-modified git source. The attacker could have used a compromised maintainer or bot credential, a private branch or mirror, or the reference the pipeline builds from. A stolen PyPI token used from the attacker’s own machines fits the evidence less well.
Makes sense. It was that a maintainer or the private pipeline was attacked and the malicious dependency was injected there by squatting on the maintainer's local copy rather than the dependency being hijacked for everyone. Very clever.
> Was `cook.pth` a dependency that got automatically downloaded from PyPI on install and then the build system put it into dependencies?
This isn't the name of an entire package, but just a single file included in the package. .pth files are part of Python's import system. At startup, the standard library `site` runs automatically, and configures the (initial) places that Python will look for code to satisfy `import` statements. By design (because some people occasionally find it very useful), arbitrary Python code can be run from these[0]. The system searches the entire site-packages folder for these files.
The compromised build packages up a .pth file in such a way that installers will treat it as a "data file"; but because Python programmers are accustomed to looking for data files at paths relative to the current source file, installers put .pth files in places that will easily be picked up by the `site` scan.
So the point is simply that, once LiteLLM is installed, malware injected directly into the final package (not a dependency) runs at startup in the compromised environment even if you don't use the LiteLLM code explicitly.
Nothing "creates a directory from the build file". The attacker-modified source includes the entire "cook" payload directory and also modifies the build file so that the build system would include its contents. No dependencies are involved (unless you're thinking of LiteLLM as being a dependency for something else).
The Python ecosystem has been working on mitigations for this. Starting in 3.15 the .pth file system is being reworked[1], but a deprecation period is necessary. Also, the PyPI repository has an opt-in system called "Trusted Publishing"[2] which is supposed to allow verifying that a wheel was built from the appropriate source directly using GitHub's CI mechanism. (Opt-in because of the long tail of developers who don't use this or any other CI system and shouldn't be expected to learn them for a small hobby project.) However, since the attacker did use the CI system from a separate GitHub repository, and compromised multiple maintainer accounts, I imagine it was possible to defeat that protection as well.
If you don't need the `site` mechanism for your project (of course, this needs to be tested per project), it can be disabled by passing the -S flag to Python[3]. This has always been available, but awareness seems to be low.
Historically, Python packaging has overall had a pretty lax security posture, and new attempts to improve that are often delayed by the consensus process and by backward compatibility concerns.
[0]: It's possible to put arbitrary code directly in the file, but rather unwieldy. Interestingly, this payload is neatly organized: the .pth file simply runs an import statement and appears to keep hundreds of lines of code in two actual Python files: one that works directly, and one that sets up a sub-hook called `sitecustomize` (another explicit provision of the `site` configuration system).
188 comments
[ 2.0 ms ] story [ 21.6 ms ] thread1. Google Cloud Outage [0]
2. eMudhra CA private key leak [1]
3. PingCap TiDB 10.6 TB Leak [2]
4. PHP2Shell CVEs [3]
and now we have Round 2 of LiteLLM after the first compromise in [4]...
[0] https://news.ycombinator.com/item?id=50163961
[1] https://news.ycombinator.com/item?id=50166579
[2] https://news.ycombinator.com/item?id=50173716
[3] https://news.ycombinator.com/item?id=50180836
[4] https://news.ycombinator.com/item?id=47501426
However, if anyone is looking for 20x faster and less "targeted" LiteLLM alternative - I'm work on one in Go. That's why GoModel it's called. https://github.com/ENTERPILOT/GoModel/
I myself have written a small llm proxy in Go, doing nothing but injecting the real API key to outgoing requests while itself validating virtual keys it generates, and it works great.
- being resilient to upstream provider downtime
- centrally managed config/controls
- visibility and allocation into usage and costs
We are deploying a gateway at work for these three anyhow. At home, GoModel's "virtual models" have been super helpful for testing out new open weights. I can change the model deployed in vLLM without having to change any of my downstream tools, which get a stable name.
https://gomodel.enterpilot.io/docs/features/virtual-models
I’m unsure how to automate this properly, maybe an LLM is the best starting point.
Ultimately, you need someone to verify releases and that probably means paying for it. I know Sonatype paid solutions utilise their own internal research to allow/block new releases.
We did a small survey of major incidents last year after shai-hulud and came to 2d cooldowns through that
(This is a situation where everybody benefits from high quality data. If you have data that contradicts mine, please share it!)
[1]: https://blog.yossarian.net/2025/11/21/We-should-all-be-using...
"Installing the Aikido Safe Chain is easy with the installation script.
curl -fsSL https://github.com/AikidoSec/safe-chain/releases/download/1.... -o /tmp/install-safe-chain.sh \ && echo "9e4af7107f06d9c0d758cb08fb36804c2af88e24972f3a9d3f8c2bd0ca002adf /tmp/install-safe-chain.sh" | sha256sum -c - \ && sh /tmp/install-safe-chain.sh \ && rm /tmp/install-safe-chain.sh "
(facepalm)
/s
Agents and malware will eradicate all goodwill left.
Good by internet.
Tell em Tom Zubiri sent ya (I don't get a commission, just good karma)
I use it, I don't cooldown, proper version pinning for me.
It's really not as bad as the accumulated headlines and exploitative hype make it sound.
It's similar to the mass hysteria and moral panics over things like child abduction. Media reports make certain dangers seem much more important than they really are, when considered in context.
> OSS Users > For the time being, please pin your version to 1.106.2 or any explicit version published prior to now.
> Enterprise Customers > There currently exists a private LiteLLM registry for enterprise customers, in which we are distributing an embargoed release that remediates a vulnerability whose disclosure date is scheduled for October 27. Our private registry is not affected. If you are an enterprise customer pulling from our private registry you are not affected.
Seems a little unethical
1. there is a second vulnerability that will be disclosed next Tuesday (can't be this one because talking about it as undisclosed in the GH issue disclosing it makes no sense)
2. the patch for that vulnerability is somewhere in the diff between $current_public_release and $current_enterprise_release
3. You have to pay to get $current_enterprise_release
They might not be selling the vulnerability directly, but they are selling access to the patch that fixes it. Which would be enough of a hint even in pre-LLM times. And unlike other vendors they don't just claim "you get faster updates and additional features" or something like that, they directly point out that if you buy now you get advance access to the fix to an unannounced vuln
I'm not saying they are advertising this to bad actors. But if they wanted to do that with a shred of plausible deniability, this would be the way to do it
There exists a critical vulnerability which affects only Enterprise LiteLLM deployments, and does not affect OSS deployments.
We have provided our customers with patched releases distributed via a private registry and are allowing them some time to update their deployments before we release the fix in our public registry and disclose the vulnerability details. Releasing the fix publicly in an open source product would be a de-facto vulnerability disclosure.
All I read was that it creates a directory from the build file but seemingly no description how new files or added dependencies slipped by. Was `cook.pth` a dependency that got automatically downloaded from PyPI on install and then the build system put it into dependencies?
--
Leaving this for others:
> BerriAI’s private release pipeline built these wheels from an attacker-modified git source. The attacker could have used a compromised maintainer or bot credential, a private branch or mirror, or the reference the pipeline builds from. A stolen PyPI token used from the attacker’s own machines fits the evidence less well.
Makes sense. It was that a maintainer or the private pipeline was attacked and the malicious dependency was injected there by squatting on the maintainer's local copy rather than the dependency being hijacked for everyone. Very clever.
This isn't the name of an entire package, but just a single file included in the package. .pth files are part of Python's import system. At startup, the standard library `site` runs automatically, and configures the (initial) places that Python will look for code to satisfy `import` statements. By design (because some people occasionally find it very useful), arbitrary Python code can be run from these[0]. The system searches the entire site-packages folder for these files.
The compromised build packages up a .pth file in such a way that installers will treat it as a "data file"; but because Python programmers are accustomed to looking for data files at paths relative to the current source file, installers put .pth files in places that will easily be picked up by the `site` scan.
So the point is simply that, once LiteLLM is installed, malware injected directly into the final package (not a dependency) runs at startup in the compromised environment even if you don't use the LiteLLM code explicitly.
Nothing "creates a directory from the build file". The attacker-modified source includes the entire "cook" payload directory and also modifies the build file so that the build system would include its contents. No dependencies are involved (unless you're thinking of LiteLLM as being a dependency for something else).
The Python ecosystem has been working on mitigations for this. Starting in 3.15 the .pth file system is being reworked[1], but a deprecation period is necessary. Also, the PyPI repository has an opt-in system called "Trusted Publishing"[2] which is supposed to allow verifying that a wheel was built from the appropriate source directly using GitHub's CI mechanism. (Opt-in because of the long tail of developers who don't use this or any other CI system and shouldn't be expected to learn them for a small hobby project.) However, since the attacker did use the CI system from a separate GitHub repository, and compromised multiple maintainer accounts, I imagine it was possible to defeat that protection as well.
If you don't need the `site` mechanism for your project (of course, this needs to be tested per project), it can be disabled by passing the -S flag to Python[3]. This has always been available, but awareness seems to be low.
Historically, Python packaging has overall had a pretty lax security posture, and new attempts to improve that are often delayed by the consensus process and by backward compatibility concerns.
[0]: It's possible to put arbitrary code directly in the file, but rather unwieldy. Interestingly, this payload is neatly organized: the .pth file simply runs an import statement and appears to keep hundreds of lines of code in two actual Python files: one that works directly, and one that sets up a sub-hook called `sitecustomize` (another explicit provision of the `site` configuration system).
[1]: https://docs.python.org/3/whatsnew/3.15.html#whatsnew315-sta...
[2]: https://docs.pypi.org/trusted-publishers/
[3]: https://docs.python.org/3/using/cmdline.html#cmdoption-S
1792590232 | Litellm 1.107.0 Compromised | https://github.com/BerriAI/litellm/issues/47826 | https://news.ycombinator.com/item?id=50180845 | 0 comments
> No public commit in BerriAI/litellm contains data = "cook" or a cook/ folder.
> SafeDep assesses, with moderate to high confidence, that BerriAI’s private release pipeline built these wheels from an attacker-modified git source.