I think it's funny that I don't see any findings from either Claude or DataDog that couldn't be detected using static analysis. They're pretty simple code bases and maybe that's why.
I'll pay more attention when they start finding vulnerabilities in commonly used, more complex applications.
Here's a better option -- what we've been working on at Snyk.
- Take something like Cursor and plug the Snyk MCP server into it: https://docs.snyk.io/integrations/developer-guardrails-for-a... (it has a one-click install)
- Then, either within your project or via global settings, create some human-language rules for your AI code editor to use (this works basically the same between all editors: Claude Code, Cursor, Windsurf, etc...)
For example, a rule might state:
"If you add or change any code, run a Snyk Code scan on the modified files then fix the detected vulnerabilities. When you're done fixing them, perform another scan to ensure they're fixed, and if not, keep iterating until the code is secure."
Obviously, there are other rules you can use here, such as using Snyk's open source dependency testing to identify vulns in third-party dependencies and handle package updates/rewrites/etc., but you get the idea.
This works insanely well -- I've been playing around with it for a while now and we're getting close to rolling this out to all of our users in a major way =)
The best part about it is that you can just "vibe code" whatever you want, and you get really accurate static analysis security testing incorporated by default automagically.
I recorded a little video here that walks through this in-depth (https://www.youtube.com/watch?v=hQtgR1lTPYI), if you want to see the part I'm referencing, jump to 20:09 =)
12 comments
[ 4.9 ms ] story [ 44.8 ms ] threadI'll pay more attention when they start finding vulnerabilities in commonly used, more complex applications.
So please, don’t be too loud about how terrible it is :)
I think an underappreciated use case for LLMs is to contextualize security issues.
Rather than asking Claude to detect problems, I think it’s more useful to let it figure out the context around vulnerabilities and help triage them.
(for better or worse, I am knee-deep in this stuff)
- Take something like Cursor and plug the Snyk MCP server into it: https://docs.snyk.io/integrations/developer-guardrails-for-a... (it has a one-click install) - Then, either within your project or via global settings, create some human-language rules for your AI code editor to use (this works basically the same between all editors: Claude Code, Cursor, Windsurf, etc...)
For example, a rule might state:
"If you add or change any code, run a Snyk Code scan on the modified files then fix the detected vulnerabilities. When you're done fixing them, perform another scan to ensure they're fixed, and if not, keep iterating until the code is secure."
Obviously, there are other rules you can use here, such as using Snyk's open source dependency testing to identify vulns in third-party dependencies and handle package updates/rewrites/etc., but you get the idea.
This works insanely well -- I've been playing around with it for a while now and we're getting close to rolling this out to all of our users in a major way =)
The best part about it is that you can just "vibe code" whatever you want, and you get really accurate static analysis security testing incorporated by default automagically.
I recorded a little video here that walks through this in-depth (https://www.youtube.com/watch?v=hQtgR1lTPYI), if you want to see the part I'm referencing, jump to 20:09 =)