ggattip
- Karma
- 0
- Created
- ()
- Submissions
- 0
- Show HN: I benchmarked LLM agents on fixing real-world security vulnerabilities (giovannigatti.github.io)
I built a benchmark with 20 real CVEs across 18 Python projects (Pillow, GitPython, yt-dlp, urllib3, etc). I've run it over 5 LLM agents (3 OpenAI, 2 poolside) and 3 different prompts (full advisory, locate, diagnose)…