HN Explorer

Show HN: TheAuditor – Offline security scanner for AI-generated code (github.com)

13 points by ThailandJohn ↗ HN
I'm an infrastructure architect who started using AI assistants to write code 3 months ago. After building several systems with Claude, I noticed a pattern: the code always had security issues I could spot from my ops background, but I couldn't fix them myself since I can't actually write code.

Why I built this: I needed a way to verify AI-generated code was production-safe. Existing tools either required cloud uploads (privacy concern) or produced output too large for AI context windows. TheAuditor solves both problems - it runs completely offline and chunks findings into 65KB segments that fit in Claude/GPT-4 context limits.

What I discovered: Testing on real projects, TheAuditor consistently finds 50-200+ vulnerabilities in AI-generated code. The patterns are remarkably consistent: - SQL queries using f-strings instead of parameterization - Hardcoded secrets (JWT_SECRET = "secret" appears in nearly every project) - Missing authentication on critical endpoints - Rate limiting using in-memory storage that resets on restart

Technical approach: TheAuditor runs 14 analysis phases in parallel, including taint analysis (tracking data from user input to dangerous sinks), pattern matching against 100+ security rules, and orchestrating industry tools (ESLint, Ruff, MyPy, Bandit). Everything outputs to structured JSON optimized for LLM consumption.

Interesting obstacle: When scanning files with vulnerabilities, antivirus software often quarantines our reports because they contain "malicious" SQL injection patterns - even though we're just documenting them. Had to implement pattern defanging to reduce false positives.

Current usage: Run aud full in any Python/JS/TS project. It generates a complete security audit in .pf/readthis/. The AI can then read these reports and fix its own vulnerabilities. I've seen projects go from 185 critical issues to zero in 3-4 iterations.

The tool is particularly useful if you're using AI assistants for production code but worry about security. It provides the "ground truth" that AI needs to self-correct.

Would appreciate feedback on: - Additional vulnerability patterns common in AI-generated code - Better ways to handle the antivirus false-positive issue - Integration ideas for different AI coding workflows

Thanks for taking a look! /TheAuditorTool

7 comments

[ 6.6 ms ] story [ 28.3 ms ] thread
quibono
> Don't create a venv before installing TheAuditor

That's a strange ask in the Python ecosystem - what's the reason for this?

Also, what's the benefit of ESLint/Ruff/MyPy being utilised by this audit tool? I'm not sure I understand the benefit of having an LLM in between you and Ruff, for example.

antonly
> TheAuditor solves ALL of this. It's not a "nice to have" - it's the missing piece that makes AI development actually trustworthy.

> I've built the tool that makes AI assistants production-ready. This isn't competing with SonarQube/SemGrep. This is creating an entirely new category: AI Development Verification Tools.

Wow, that's a lot of talk for a tool that does regex searches and some AST matching, supporting only python and js (these things are not mentioned in the main project README as far as I can tell?).

The actual implementation details are buried in an (LLM written?) document: https://github.com/TheAuditorTool/Auditor/blob/main/ARCHITEC...

My favourite part is the "Pipeline System", which outlines a "14-phase analysis pipeline", but does not number these stages.

It reads a bit like the author is hiding what the tool actually does, which is sad, because there might be some really neat ideas in there, but they are really hard to make out.

iamsaitam
"This perfectly illustrates why I need community input. I'm not a developer - I literally can't code. I built this entire tool using Claude over 250 hours because I needed something to audit the code that Claude was writing for me. It's turtles all the way down!" - should be in bold on a huge banner
grim_io
Using an established analysis tool like sonarcube is probably the way to go.

There is no difference between human made and AI made bad code, so I don't think we need specialized tools for that.

enjoytheview
A security project vibe coded by someone who admittedly does not have a security or even software engineering background, what could go wrong!
lewdwig
I have noticed that LLMs are actually pretty decent at redteaming code, so I’ve made it a habit of getting them to do that for code they generate periodically. A good loop is (a) generate code, (b) add test coverage for the code (to 70-80%) (c) redteam the code for possible performance/security concerns, (d) add regression tests for the issues uncovered and then fix the code.
pogue
Can someone recommend some alternatives to a tool like this?