I’ve been working with ML infrastructure for a while and realized there’s a gap in the security posture: we scan our requirements.txt for vulnerabilities, but blindly trust the 5GB binary model files (.pt) we download from Hugging Face.
Most developers don't realize that standard PyTorch files are just Zip archives containing Python Pickle bytecode. When you run torch.load(), the unpickler executes that bytecode. This allows for arbitrary code execution (RCE) inside the model file itself - what security researchers call a "Pickle Bomb."
I built AIsbom (AI Software Bill of Materials) to solve this without needing a full sandbox.
How it works:
1. It inspects the binary structure of artifacts (PyTorch, Pickle, Safetensors) without loading weights into RAM.
2. For PyTorch/Pickles, it uses static analysis (via pickletools) to disassemble the opcode stream.
3. It looks for GLOBAL or STACK_GLOBAL instructions referencing dangerous modules like os.system, subprocess, or socket.
4. It outputs a CycloneDX v1.6 JSON SBOM compatible with enterprise tools like Dependency-Track.
5. It also parses .safetensors headers to flag "Non-Commercial" (CC-BY-NC) licenses, which often slip into production undetected.
You asked for specific feedback, but here is generic feedback: a new github account coupled to a new HN account does not inspire any sense of added infra safety. I would rather use modern pytorch/safetensors and tools that dont allow executing pickles from checkpoints. If you execute someone elses pickle you probably already lost no matter what checks you want to add over time.
> but blindly trust the 5GB binary model files (.pt) we download from Hugging Face.
I thought the ecosystem had mostly moved to .safetensors (which was explicitly created to fix this problem) and .gguf (which I'm pretty sure also doesn't have this problem); do you really need to download giant chunks of untrusted code and execute it at all?
Thanks for sharing this — really solid write-up, and I agree with the core premise. Pickle is a huge blind spot in ML security, and most folks don’t realize that torch.load() is effectively executing attacker-controlled bytecode.
One thing we ran into while working on similar problems is that static opcode scanning alone tends to give a false sense of coverage. A lot of real-world bypasses don’t rely on obvious GLOBAL os.system patterns and can evade tools that depend on pickletools, modelscan, or fickling.
We recently open-sourced a structure-aware pickle fuzzer at Cisco that’s designed specifically to test the robustness of pickle scanners, not just scan models:
• It executes pickle bytecode inside a custom VM, tracking opcode execution, stack state, and memo behavior
• Mutates opcode sequences, stack interactions, and protocol-specific edge cases
• Has already uncovered multiple scanner bypasses that look benign statically but behave differently at runtime
I think tools like AIsbom are a great step forward, especially for SBOM and ecosystem visibility. From our experience, pairing static analysis + fuzzing-driven adversarial testing is where things get much more resilient over time.
This is incredibly valuable feedback. I’ve been reading through the pickle-fuzzer repo this morning, specifically about stack manipulation bypassing static heuristics.
You nailed the trade-off: AIsbom is designed for the "90% hygiene" case in a fast CI/CD pipeline (where spinning up a VM/Fuzzer might be too heavy/slow for every commit). We aim to catch the low-hanging fruit (obvious RCE) and generate the Inventory (SBOM) rapidly.
That said, moving toward an "Allowlist Only" (Strict Mode) approach seems like the better way to make static analysis resilient against the obfuscation you mentioned. We are prioritizing that for upcoming release. Would love to potentially reference your fuzzer in our docs as the "Deep Scan" alternative!
When dealing with stuff like php serialization and pickle, the rule is simple: never unpickle anything you didn't pickle yourself. If anything else could possibly touch the serialized bytes, sign it with HMAC and keep that somewhere untouchable.
I somehow doubt this tool is going to be able to pull off what Java bytecode verification could not.
The checks here seem pretty minimal[1]. I'd recommend taking a look at fickling (FD: former employer) for a more general approach to pickle decompilation/analysis[2].
Ok, as others noted, the tool in question is hardly a solution, but what is, then? I mean, presented like that, it's pretty crazy that everyone just downloads and runs 5GB executable blobs from Hugging Face. Does anyone review them somehow before they are accepted and gather 10K downloads on HF? Or is it really another totally mindbogglingy crazy thing that happens right now across all world, and everybody just shrugs and waits for catastrophic breach of security of planetary scale to happen?
Was there any research into prior art? Recently did some research into this space and it seems like there are already a bunch of off the shelf open source projects for address this
16 comments
[ 4.0 ms ] story [ 38.2 ms ] threadI’ve been working with ML infrastructure for a while and realized there’s a gap in the security posture: we scan our requirements.txt for vulnerabilities, but blindly trust the 5GB binary model files (.pt) we download from Hugging Face.
Most developers don't realize that standard PyTorch files are just Zip archives containing Python Pickle bytecode. When you run torch.load(), the unpickler executes that bytecode. This allows for arbitrary code execution (RCE) inside the model file itself - what security researchers call a "Pickle Bomb."
I built AIsbom (AI Software Bill of Materials) to solve this without needing a full sandbox.
How it works: 1. It inspects the binary structure of artifacts (PyTorch, Pickle, Safetensors) without loading weights into RAM. 2. For PyTorch/Pickles, it uses static analysis (via pickletools) to disassemble the opcode stream. 3. It looks for GLOBAL or STACK_GLOBAL instructions referencing dangerous modules like os.system, subprocess, or socket. 4. It outputs a CycloneDX v1.6 JSON SBOM compatible with enterprise tools like Dependency-Track. 5. It also parses .safetensors headers to flag "Non-Commercial" (CC-BY-NC) licenses, which often slip into production undetected.
It’s open source (Apache 2.0) and written in Python/Typer. Repo: https://github.com/Lab700xOrg/aisbom Live Demo (Web Viewer): https://aisbom.io
Why I built a scanner? https://dev.to/labdev_c81554ba3d4ae28317/pytorch-models-are-...
I’d love feedback on the detection logic (specifically safety.py) or if anyone has edge cases of weird Pickle protocols that break the disassembler.
is anyone calling it that? to me, "pickle bomb" would imply abusing compression or serialization for a resource-exhaustion attack, a la zipbombs.
"pickle bomb", the way you're using it, doesn't seem like a useful terminology -- pickles are just (potentially malicious) executables.
I thought the ecosystem had mostly moved to .safetensors (which was explicitly created to fix this problem) and .gguf (which I'm pretty sure also doesn't have this problem); do you really need to download giant chunks of untrusted code and execute it at all?
This is outrageous. Why not deprecate this cursed format and use something from the data frame community? Like, Parquet or something
Actually almost any binary format is better than this
One thing we ran into while working on similar problems is that static opcode scanning alone tends to give a false sense of coverage. A lot of real-world bypasses don’t rely on obvious GLOBAL os.system patterns and can evade tools that depend on pickletools, modelscan, or fickling.
We recently open-sourced a structure-aware pickle fuzzer at Cisco that’s designed specifically to test the robustness of pickle scanners, not just scan models:
• It executes pickle bytecode inside a custom VM, tracking opcode execution, stack state, and memo behavior • Mutates opcode sequences, stack interactions, and protocol-specific edge cases • Has already uncovered multiple scanner bypasses that look benign statically but behave differently at runtime
Repo: https://github.com/cisco-ai-defense/pickle-fuzzer
We also wrote up some of the lessons learned while hardening pickle scanners here (including why certain opcode patterns are tricky to reason about statically): https://blogs.cisco.com/ai/hardening-pickle-file-scanners
I think tools like AIsbom are a great step forward, especially for SBOM and ecosystem visibility. From our experience, pairing static analysis + fuzzing-driven adversarial testing is where things get much more resilient over time.
I somehow doubt this tool is going to be able to pull off what Java bytecode verification could not.
[1]: https://github.com/Lab700xOrg/aisbom/blob/main/aisbom/safety...
[2]: https://github.com/trailofbits/fickling