HN Explorer

Show HN: AI Code Detector – detect AI-generated code with 95% accuracy (code-detector.ai)

72 points by henryl ↗ HN
Hey HN,

I’m Henry, cofounder and CTO at Span (https://span.app/). Today we’re launching AI Code Detector, an AI code detection tool you can try in your browser.

The explosion of AI generated code has created some weird problems for engineering orgs. Tools like Cursor and Copilot are used by virtually every org on the planet – but each codegen tool has its own idiosyncratic way of reporting usage. Some don’t report usage at all.

Our view is that token spend will start competing with payroll spend as AI becomes more deeply ingrained in how we build software, so understanding how to drive proficiency, improve ROI, and allocate resources relating to AI tools will become at least as important as parallel processes on the talent side.

Getting true visibility into AI-generated code is incredibly difficult. And yet it’s the number one thing customers ask us for.

So we built a new approach from the ground up.

Our AI Code Detector is powered by span-detect-1, a state-of-the-art model trained on millions of AI- and human-written code samples. It detects AI-generated code with 95% accuracy, and ties it to specific lines shipped into production. Within the Span platform, it’ll give teams a clear view into AI’s real impact on velocity, quality, and ROI.

It does have some limitations. Most notably, it only works for TypeScript and Python code. We are adding support for more languages: Java, Ruby, and C# are next. Its accuracy is around 95% today, and we’re working on improving that, too.

If you’d like to take it for a spin, you can run a code snippet here (https://code-detector.ai/) and get results in about five seconds. We also have a more narrative-driven microsite (https://www.span.app/detector) that my marketing team says I have to share.

Would love your thoughts, both on the tool itself and your own experiences. I’ll be hanging out in the comments to answer questions, too.

31 comments

[ 4.0 ms ] story [ 48.0 ms ] thread
bigyabai
I can detect AI-generated code with 100% accuracy, provided you give me an unlimited budget for false positives. It's a bit of a useless metric.
samfriedman
Accuracy is a useless statistic: give us precision and recall.
JohnFriel
This is interesting. Do you know what features the classifier is matching on? Like how much does stuff like whitespace matter here vs. deeper code structure? Put differently, if you were to parse the AI and non-AI code into AST and train a classifier based on that, would the results be the same?
Ndotkess
What is your approach to measuring accuracy?
jensneuse
Could I use this to iterate over my AI generated code until it's not detectable anymore? So essentially the moment you publish this tool it stops working?
[deleted]
(comment deleted)
icemanx
Would be amazing to have a CLI tool that detects AI generated code (even add it as part of CI/CD pipelines). I'm tired of all the AI trash PRs
mechen
As a leader this is actually really neat - going to give it a spin
Alifatisk
Very cool piece of tech, I would suggest putting C on the priority list and then Java. Mainly because Unis and Colleges use one of them or both, so that would be a good use case
mechen
Just tried it out and it works :mind-blown:
p0w3n3d
I wonder how many false positives it has
jjmarr
You're saying "Understand and report on impact by AI coding tool". How can you drill down into per-coding assistant usage?

Also, what's the pricing?

mendeza
I feel like code fed into this detector can be manipulated to increase false positives. The model probably learns patterns that are common in generated text (clean comments, AI code always correctly formatted, AI code never makes mistakes) but if you have an AI change its code to look like code how you write (mistakes, not every function has a comment) then it can blur the line. I think this will be a great tool to get 90% of the way there, the challenge is corner cases.
fancyfredbot
An AI code detector would be a binary text classifier - you input some text and the output is either "code" or "not-code".

This is an "AI AI code detector".

You could call it a meta-AI code detector but people might think that's a detector for AI code written by the company formerly known as Facebook.

jftuga
I will always write code myself but then sometimes have AI generate a first pass at class and method doc strings. What would happen in this scenario with your tool? Would my code be detected as AI generated because of this or does your tool solely operate on code only?
ht-syseng
Just tried. Actually quite impressed with how well it works. I avoid using AI to write code, I'm a little worried that the existence of detection tools like this will lead people to over-rely on them; I would feel bad if someone suggested I used AI to create code I took pride in writing. I don't matter, but on a societal scale that effect may compel people to over-rely on AI as their work is treated as slop whether they put effort in or not, which will just increase the tide of terrible AI slop code, engineers managing systems they do not understand, and thus the brittleness and instability of global infrastructure. I sincerely hope you guys succeed, I suppose the point is that almost succeeding might be worse than not trying at all...
triwats
Firstly I think this is neat, but the dam has burst.

This might be great for educational institutions but the idea of people needing to know what everyline does as output feels mute to me in the face of agentic AI.

jimmydin7
what will the pricing be? i guess this is just a super early demo, I want to hear your pricing plan. Also, is this B2B or B2C?
[deleted]
(comment deleted)
jakderrida
What if I just modify the code to misspell things that no AI would misspell?
khanna_ayush
My engineers didn’t know how much they used AI for vibe coding until I used Span. Can confirm we were all left with jaws on the floor. Now re-thinking my hiring plan for the next year.
kittikitti
A 95% accuracy is very low for this type of thing. People use this to enact administrative consequences. People's lives are ruined and 5% is too high of a false positive rate. Even a 99% accuracy is too low.