Semgrep started off as a “syntactic grep” but has increasingly become more semantic. So if you want to find all calls to foo that have 1 as the first argument, you just search for foo(1) and even things like x = 1; foo(x); will match.
It does seem potentially good for enforcing standards where the participants are willing. But you can work around it fairly easily. Like the example "python no-prints" rule: https://semgrep.dev/s/sabihb:no-prints
I'm guessing the value with the rules system is, you can add new rules easily. So during a code review, if you see somebody using your example, you could create a new rule to catch that.
I don't think you can go in with the mindset that it will catch everything, but rather, it's about being able to iterate quickly with your rules.
Sure. I raised it because it keeps using the word "static analysis", which at least wouldn't be fooled by builtins.print(). It's more than grep, for sure, but something less than static analysis tools I've used. And I don't mean that as a knock. It's working across a lot of languages, so I see the tradeoff.
Cool! But this example is a bit simplistic since it can be done just as easily by regular grep:
grep -E '(.+) = \1' *.py
I have trouble looking at the examples in the project website (many things inside iframes are adblocked). Do you have any example of a search that would be difficult or impossible with grep?
If you're that capable with grep, surely you've run into it's shortcomings and can appreciate attempts to improve the status quo. Like trying to find usages of a variable that is a substring of a word. Like trying to fix code where somebody else has used the variable name i. Sure, you can add matching on word boundaries, but then it starts to become a distraction from the original task at hand.
to follow myself up, one can also check for expressions like what I used as an example and say "don't do that", but without regard to what is inside the if
Hey, I work on Semgrep. As a real world example, I just noticed today that Hashicorp uses a whole bunch of Semgrep rules on terraform-provider-aws[0]. I'd recommend reading the `message` keys to know what they intend to match, and then the `patterns` lists below to see how that's accomplished.
Alternatively, we curate 1000+ community rules that you can look through as well.[1]
Nice! Thanks. This will certainly help me start to thinking in semantic grep. I can see this being an additional coverage tool and am eager to study it.
This is an excellent tool to have as a security consultant, and it just keeps getting better and better. When approaching a large codebase, it enables you to write custom rules that match on certain antipatterns you've spotted that may be unique to the codebase. That's the real value of the tool, but the repository of per-language rules is also convenient for quickly finding low-hanging fruit (like every use of a potentially injectable function such as exec,system,etc. in PHP).
For example, a webapp may have been designed such that authorisation needs to be explicitly added with a line or two to each controller. A semgrep rule can be written to match all the controllers which are missing this line. Then these controllers can be manually reviewed to assess whether unauthorised access should be allowed. Depending on what you are trying to match, this is something that may be very complex or even impossible to implement accurately in plain grep. Some languages like Ruby have powerful static analysis tools (Brakeman) that can also do this, but the benefit of Semgrep is the flexibility across multiple languages and how readable the rulesets are. [1]
Perhaps, but this tool has a syntax and common interface for multiple languages. This is a huge resource for IT/security. I could imagine this being a helpful refactoring tool as well for CLI.
I'd be careful with how much of a warm fuzzy the tool gives you. See this example from my other comment in the thread: https://news.ycombinator.com/item?id=26905880 If it were really looking at AST level data, that wouldn't have fooled it.
I suspect there would be similar issues with your example of ensuring no use of eval() in PHP. So it seems okay to keep your own developers informed, but I wouldn't use it, alone, to vet outside code. PHP has eval-like functionality buried in preg_replace(), assert(), and probably other places. This tool also doesn't seem to dig into namespaced "aliases".
> If it were really looking at AST level data, that wouldn't have fooled it.
Semgrep does look at an AST; but that counterexample is not something you can "fix" solely by looking at an AST. You need actual Python-specific semantic analysis that knows that all "open" functions like 'print' come from the builtins module, and thus are bound to the same identifier. They're literally built into the implementation, it's not something you can "discover" from analyzing existing Python source. Even if you had a perfectly accurate python AST it couldn't "tell" you this fact, it's a priori knowledge, and all analysis engines need a base set of facts like this that they work from.
> but I wouldn't use it, alone, to vet outside code.
I mean, nobody seems to be suggesting this though, and the OP quite literally stated the major value of the tool is enforcing domain/codebase-specific rules among a team. Which is a really good use for it! There are tons of little useful patterns you can codify this way.
>Semgrep does look at an AST; but that counterexample is not something you can "fix" solely by looking at an AST
Perhaps I worded it poorly. Dumping the python AST for builtins.print() makes it pretty clear that it's "print" though. So I'm curious why that skirts the rule.
>I mean, nobody seems to be suggesting this though
Not specificially, but the context is using it for security purposes with phrases like "every use of a potentially injectable function such as exec,system,etc. in PHP". Felt like that was worth commenting on.
> Dumping the python AST for builtins.print() makes it pretty clear that it's "print" though.
No, the AST only tells you it's a method call on something called "builtins." You need the separate semantic knowledge of what builtins is in order to figure it out. Parsing + AST just means it sees "method call of `print` on `builtins` object". Regular print calls would come through as "regular function call of `print`".
> Perhaps I worded it poorly. Dumping the python AST for builtins.print() makes it pretty clear that it's "print" though. So I'm curious why that skirts the rule.
To echo the other replies, the AST for builtins.print() is the same as the ast for mymodule.print() and, in fact, if you stick a builtins.py in the right place, you'll be able to prevent the import of the standard library builtin module, while the ast's would be identical.
I'm honestly surprised we haven't seen complete democratization of linting/grepping/refactoring tools. Imagine being able to script a one-off refactor you want to make that's too project-specific to be included in the IDE itself.
Writing the parser is nontrivial, but once you have it it should be straightforward to expose a programmatic API for doing this stuff instead of trying to hardcode every useful linter rule into a single program.
We do this with rubocop and its ability to parse the ruby AST. The API to actually write rules in rubocop is not particularly the best, but it has been highly successful in codebase and domain-specific ruby linting and autofixing.
Writing a parser for one specific version of a language is one thing (nontrivial), but writing a parser than parsed most versions of most programming languages and keeping it up-to-date is a enormous undertaking and cannot be done by one person
For C/C++ code, you can already do refactoring using clang-tidy scripts [0], or even can write custom linters using libtooling [1] and leverage the AST Matchers [2] which work at the AST level.
All that's needed is a compile_commands.json file which can be easily generated via most build systems, or you can use Bear [3]/some other tool (or write a script that logs all syscalls and generate it yourself).
"GitLab SAST historically has been powered by over a dozen open-source static analysis security analyzers. These analyzers have proactively identified millions of vulnerabilities for developers using GitLab every month. Each of these analyzers is language-specific and has different technology approaches to scanning. These differences produce overhead for updating, managing, and maintaining additional features we build on top of these tools, and they create confusion for anyone attempting to debug.
The GitLab Static Analysis team is continuously evaluating new security analyzers. We have been impressed by a relatively new tool from the development team at r2c called Semgrep. It’s a fast, open-source, static analysis tool for finding bugs and enforcing code standards. Semgrep’s rules look like the code you are searching for; this means you can write your own rules without having to understand abstract syntax trees (ASTs) or wrestle with regexes.
Semgrep’s flexible rule syntax is ideal for streamlining GitLab’s Custom Rulesets feature for extending and modifying detection rules, a popular request from GitLab SAST customers. Semgrep also has a growing open-source registry of 1,000+ community rules.
We are in the process of transitioning many of our lint-based SAST analyzers to Semgrep. This transition will help increase stability, performance, rule coverage, and allow GitLab customers access to Semgrep’s community rules and additional custom ruleset capabilities that we will be adding in the future. We have enjoyed working with the r2c team and we cannot wait to transition more of our analyzers to Semgrep. You can read more in our transition epic, or try out our first experimental Semgrep analyzers for JavaScript, TypeScript, and Python.
We are excited about what this transition means for the future of GitLab SAST and the larger Semgrep community. GitLab will be contributing to the Semgrep open-source project including additional rules to ensure coverage matches or exceeds our existing analyzers."
Isn't a problem here that, since there's no simple definition of what it's doing, there's no simple way to assess your false negative rate when searching for something? That is fine for many pragmatic informal uses but doesn't seem a good fit for security purposes.
I clicked on this thinking it's a grep that can search code snippets based on language-aware syntax matching instead of regular expressions.
Agreed, this project name is misleading about what it does. The name "grep" always indicated some kind of "find a text/pattern and print results to stdout" utility. Like pgrep, which searches running processes by name and then prints their IDs.
> it's a grep that can search code snippets based on language-aware syntax matching instead of regular expressions.
Hey, I'm a maintainer of Semgrep, and this sounds like a pretty good description of what the CLI can do, see this example for finding all function/class/method calls:
Oh, that's great to see. The website's presentation made a different impression with its "enforce code standards" angle.
Looks like a pretty useful tool with a couple nice options. A bit strange that the `-e` option is only explained on the website, but to be fair it seems to be a lot to cover. Still, a kind of "cheat sheet" style summary in the help message would be fantastic, just as a little suggestion.
question, you mention "no more AST parsing" as a selling line for semgrep. Isn't this actually a bad thing for a staric analyzer, as ASTs are actually... abstract, and closer to the real logic of what the code does? I know analyzing text is faster but is it worth?
I think the name's just fine. It searches through your code in a way that you can define semantically/with more context. All of the alternatives you mention carry less meaning, and are less memorable.
This is like saying that PubNub is a bad name because it does messaging, and has little to do with pub-sub. Or hell, even that Y Combinator should be called something like StartupFactory since it's not really a recursive tool.
Is there a more complete example of how to call semgrep from pre-commit (which gets called before every git commit) in order to prevent e.g. Python print calls (print(), print \\n(), etc.) from being checked in?
Yeah but that githook will only be installed on that one repo on that one machine. And they may have no or a different version of bash installed (on e.g. MacOS or Windows). IMHO, POSIX-compatible portable shell scripts are more trouble than portable Python scripts.
Pre-commit requires Python and pre-commit to be installed (and then it downloads every hook function).
This fetches the latest version of every hook defined in the .pre-commit-config.yml:
Kind of crazy that you can make a tool like this in the modern age that isn't cross platform. Maybe they just can't face the Python packaging nightmare on Windows.
Also kind of surprising it's written in Python given that they advertise its speed.
I’m always surprised at stuff I take for granted that doesn’t work on Windows. So yeah, it seems like cross-platform should be easy but since my dev environment is zsh, it’s easy for my stuff to work sort of “everywhere but Windows.”
Add to that that the reason things fail on Windows is usually something Windows specific and “their fault.” So it’s unusual for me to fire up a Windows VM just to sanity check my code. And since Windows CI runners cost 2x or more, I don’t usually run cross platform CI.
Windows runners consume 2x minutes, Mac runners consume 10x minutes [0].
Free projects only get 2,000 minutes per month so running on all three platforms means you only get 1/13th of the minutes of only using Linux.
I rarely use anything but Linux runners even on my paid projects. I like saving money, so unless I really need integration testing on Windows or Mac, I don’t do it.
I was surprised by this too, but as a maintainer of the project we’ve had comparatively little Windows requests compared to other systems. Qualitatively, for the folks who have asked for Windows support, WSL has been a viable option and reduced the support surface area for the small team behind the tool.
On the speed front, the core of Semgrep is OCaml, with a Python wrapper to add niceties like pattern composition. I’m less close to that part of the codebase, but think more and more logic is being moved to OCaml for performance reasons.
Just the tool that I was looking for. We are looking to do Service linting in our organization as a method of making sure our services don't drift too far apart.
Anyone else know of a Service linting tool? OPA/conftest come close but lack syntax parsers for Ruby/Javascript.
I currently use a highly opinionated ESLint config (based on the airbnb one) together with strict checking in my TypeScript config, and it is configured to run on every commit with husky git hooks. The example given on the Semgrep homepage is an exact match to one that exists in my ESLint config (eslint's no-console rule).
How does Semgrep compare to ESLint+a strict tsconfig?
If the commit hook rejects anything where rules are triggered, a way to force the comrit is needed for the cases when the rule finding is not reanny an issue.
Upd: I found that the --no-verify option can be used in many cases
It seems from reading the docs that the “semantic” side is important here. It can track things like redefinitions.
E.g.
const output = console.log;
output(“hello”);
This wouldn’t be caught by ESLint (to my knowledge) but would be caught by Semgrep. I think you could do it with ESLint but given the interface for an ESLint plugin exposes an AST, you’d have to track this yourself. I’m assuming Semgrep could stretch to things like enforcing APIs are called with certain optional arguments present (even if the TS types don’t require it). Again, I think with ESLint you’d have to do more juggling with the raw AST.
That said, this is my understanding from a quick skim!
I used to use SAST-SCAN but that seems abandonware. I like that this exists. Everyone should go from nothing to something in the SAST space. A free/freemium tool/service for that is pretty great. The first couple runs have found useful results.
Does it come with a standard set of rules that finds bad code without any false positives out of the box? Or is it more of a tool for people doing code security audits & pentesting who know what they are looking for and want to read the surrounding code?
Hi, this is very cool. I have been building up a suite of tools to roll out across major open source projects to improve security. I like what I have seen so far, this is a great use case. Whom can I connect with to learn more? And similarity/diff with sourcegraph, also like a lot.
probably doing something wrong but running the ci ruleset on a tiny django hobby project made all cores spin at 100% after 33% of the progress bar and made the OS almost unresponsive. ctrl-c after 5 minutes and i still had to pkill every semgrep process... never seen the M1 airbook overheat this much before.
The CI service is free, with some limitations on how long the findings stay on the dashboard, SSO integration and maybe a few others. The paid version was $40/usr/mo the last time I checked
Once we figured it out, it takes us a few minutes to onboard a new repo to Semgrep
109 comments
[ 4.3 ms ] story [ 195 ms ] threadHere’s an elaborate example: https://semgrep.dev/s/ievans:c-dataflow
Lots of workarounds it wouldn't find, like:
I don't think you can go in with the mindset that it will catch everything, but rather, it's about being able to iterate quickly with your rules.
Wait, is this a web app? I was expecting a command line tool to navigate my code locally.
You can write stuff like
given:
then to follow myself up, one can also check for expressions like what I used as an example and say "don't do that", but without regard to what is inside the ifLike, if you've never tasted lychee, it would never occur to you how to cook with it.
I'm going to need to see some useful, real-world examples to jumpstart my brain to think this way.
Alternatively, we curate 1000+ community rules that you can look through as well.[1]
[0]: https://github.com/hashicorp/terraform-provider-aws/blob/mai...
[1]: https://semgrep.dev/r
For example, a webapp may have been designed such that authorisation needs to be explicitly added with a line or two to each controller. A semgrep rule can be written to match all the controllers which are missing this line. Then these controllers can be manually reviewed to assess whether unauthorised access should be allowed. Depending on what you are trying to match, this is something that may be very complex or even impossible to implement accurately in plain grep. Some languages like Ruby have powerful static analysis tools (Brakeman) that can also do this, but the benefit of Semgrep is the flexibility across multiple languages and how readable the rulesets are. [1]
[1] https://blog.includesecurity.com/2021/01/custom-static-analy...
I suspect there would be similar issues with your example of ensuring no use of eval() in PHP. So it seems okay to keep your own developers informed, but I wouldn't use it, alone, to vet outside code. PHP has eval-like functionality buried in preg_replace(), assert(), and probably other places. This tool also doesn't seem to dig into namespaced "aliases".
Semgrep does look at an AST; but that counterexample is not something you can "fix" solely by looking at an AST. You need actual Python-specific semantic analysis that knows that all "open" functions like 'print' come from the builtins module, and thus are bound to the same identifier. They're literally built into the implementation, it's not something you can "discover" from analyzing existing Python source. Even if you had a perfectly accurate python AST it couldn't "tell" you this fact, it's a priori knowledge, and all analysis engines need a base set of facts like this that they work from.
> but I wouldn't use it, alone, to vet outside code.
I mean, nobody seems to be suggesting this though, and the OP quite literally stated the major value of the tool is enforcing domain/codebase-specific rules among a team. Which is a really good use for it! There are tons of little useful patterns you can codify this way.
Perhaps I worded it poorly. Dumping the python AST for builtins.print() makes it pretty clear that it's "print" though. So I'm curious why that skirts the rule.
>I mean, nobody seems to be suggesting this though
Not specificially, but the context is using it for security purposes with phrases like "every use of a potentially injectable function such as exec,system,etc. in PHP". Felt like that was worth commenting on.
No, the AST only tells you it's a method call on something called "builtins." You need the separate semantic knowledge of what builtins is in order to figure it out. Parsing + AST just means it sees "method call of `print` on `builtins` object". Regular print calls would come through as "regular function call of `print`".
To echo the other replies, the AST for builtins.print() is the same as the ast for mymodule.print() and, in fact, if you stick a builtins.py in the right place, you'll be able to prevent the import of the standard library builtin module, while the ast's would be identical.
Writing the parser is nontrivial, but once you have it it should be straightforward to expose a programmatic API for doing this stuff instead of trying to hardcode every useful linter rule into a single program.
In the JVM world annotation processors or compiler plugins can have access to the AST during compilation. Both in Kotlin and Java.
All that's needed is a compile_commands.json file which can be easily generated via most build systems, or you can use Bear [3]/some other tool (or write a script that logs all syscalls and generate it yourself).
[0] https://releases.llvm.org/12.0.0/tools/clang/tools/extra/doc...
[1] https://releases.llvm.org/12.0.0/tools/clang/docs/LibTooling...
[2] https://releases.llvm.org/12.0.0/tools/clang/docs/LibASTMatc...
[3] https://github.com/rizsotto/Bear
https://github.com/elanning/checkr
It is just simple regex at this time, but hopefully I can add something like CCGrep syntax in the future:
https://github.com/yuy-m/CCGrep
"GitLab SAST historically has been powered by over a dozen open-source static analysis security analyzers. These analyzers have proactively identified millions of vulnerabilities for developers using GitLab every month. Each of these analyzers is language-specific and has different technology approaches to scanning. These differences produce overhead for updating, managing, and maintaining additional features we build on top of these tools, and they create confusion for anyone attempting to debug.
The GitLab Static Analysis team is continuously evaluating new security analyzers. We have been impressed by a relatively new tool from the development team at r2c called Semgrep. It’s a fast, open-source, static analysis tool for finding bugs and enforcing code standards. Semgrep’s rules look like the code you are searching for; this means you can write your own rules without having to understand abstract syntax trees (ASTs) or wrestle with regexes.
Semgrep’s flexible rule syntax is ideal for streamlining GitLab’s Custom Rulesets feature for extending and modifying detection rules, a popular request from GitLab SAST customers. Semgrep also has a growing open-source registry of 1,000+ community rules.
We are in the process of transitioning many of our lint-based SAST analyzers to Semgrep. This transition will help increase stability, performance, rule coverage, and allow GitLab customers access to Semgrep’s community rules and additional custom ruleset capabilities that we will be adding in the future. We have enjoyed working with the r2c team and we cannot wait to transition more of our analyzers to Semgrep. You can read more in our transition epic, or try out our first experimental Semgrep analyzers for JavaScript, TypeScript, and Python.
We are excited about what this transition means for the future of GitLab SAST and the larger Semgrep community. GitLab will be contributing to the Semgrep open-source project including additional rules to ensure coverage matches or exceeds our existing analyzers."
The web page states: "Static analysis at ludicrous speed. Find bugs and enforce code standards"
"grep" is short for "global regular expression print". It finds matches for the given regular expression and prints them.
"Semantic Grep" is a static analyzer with configurable rules, style checks, etc. It does much more than search and print.
Perhaps a better name is needed?
Edit: How about "omnilint" or "omnicritic" since semgrep is more of a "lint" (https://en.wikipedia.org/wiki/Lint_(software)) or "critic" (https://en.wikipedia.org/wiki/Perl::Critic) type of tool that handles multiple languages?
Edit2: "Static analysis at ludicrous speed" ==> "turbolint"? ("ludicrous speed" reminds of the hilarious Space Balls scene :) "turbolint, GO!"
But at least to me, semgrep looks a lot more like "lint" than "grep".
Agreed, this project name is misleading about what it does. The name "grep" always indicated some kind of "find a text/pattern and print results to stdout" utility. Like pgrep, which searches running processes by name and then prints their IDs.
Hey, I'm a maintainer of Semgrep, and this sounds like a pretty good description of what the CLI can do, see this example for finding all function/class/method calls:
Looks like a pretty useful tool with a couple nice options. A bit strange that the `-e` option is only explained on the website, but to be fair it seems to be a lot to cover. Still, a kind of "cheat sheet" style summary in the help message would be fantastic, just as a little suggestion.
This is like saying that PubNub is a bad name because it does messaging, and has little to do with pub-sub. Or hell, even that Y Combinator should be called something like StartupFactory since it's not really a recursive tool.
In short, the metaphor's close enough.
Go down, see "brew install semgrep" and try to copy paste it. And it's an image :(
Go to https://semgrep.dev/p/jwt
Go to the page 2/5
Click "Run Locally", so you can copy the code
close the modal -> you're on page 1/5. Expectation would be to stay on page 2/5.
It would also be very useful to be able to filter by language and topic.
https://semgrep.dev/docs/extensions/ describes how to do pre-commit.
Nvm, here's semgrep's own .pre-commit-config.yml for semgrep itself: https://github.com/returntocorp/semgrep/blob/develop/.pre-co...
`.git/hooks` directory in your repo for samples, e.g. `.git/hooks/pre-commit.sample`.
You can run any old shell script there, without having to install a python tool.
Pre-commit requires Python and pre-commit to be installed (and then it downloads every hook function).
This fetches the latest version of every hook defined in the .pre-commit-config.yml:
https://pre-commit.com/#pre-commit-autoupdateA person could easily `ln -s repo/.hooks/hook*.sh repo/.git/hooks/` after every git clone.
(I'm guessing from your comment that this is important to you (i.e. WSL/Docker is not a solution).)
Also kind of surprising it's written in Python given that they advertise its speed.
Add to that that the reason things fail on Windows is usually something Windows specific and “their fault.” So it’s unusual for me to fire up a Windows VM just to sanity check my code. And since Windows CI runners cost 2x or more, I don’t usually run cross platform CI.
Free projects only get 2,000 minutes per month so running on all three platforms means you only get 1/13th of the minutes of only using Linux.
I rarely use anything but Linux runners even on my paid projects. I like saving money, so unless I really need integration testing on Windows or Mac, I don’t do it.
[0] https://docs.github.com/en/github/setting-up-and-managing-bi...
On the speed front, the core of Semgrep is OCaml, with a Python wrapper to add niceties like pattern composition. I’m less close to that part of the codebase, but think more and more logic is being moved to OCaml for performance reasons.
Anyone else know of a Service linting tool? OPA/conftest come close but lack syntax parsers for Ruby/Javascript.
How does Semgrep compare to ESLint+a strict tsconfig?
If the commit hook rejects anything where rules are triggered, a way to force the comrit is needed for the cases when the rule finding is not reanny an issue.
Upd: I found that the --no-verify option can be used in many cases
E.g. const output = console.log; output(“hello”);
This wouldn’t be caught by ESLint (to my knowledge) but would be caught by Semgrep. I think you could do it with ESLint but given the interface for an ESLint plugin exposes an AST, you’d have to track this yourself. I’m assuming Semgrep could stretch to things like enforcing APIs are called with certain optional arguments present (even if the TS types don’t require it). Again, I think with ESLint you’d have to do more juggling with the raw AST.
That said, this is my understanding from a quick skim!
Still, it seems rather cool, I like the idea of being able to search code at a higher level than just raw source text.
I've found the flake8 API and documentation lacking, so perhaps just a cleaner interface?
Perhaps a more pointed set of questions: what is the error it emits, and have you considered submitting that case as an actual bug?