I'm loving the graphs which for the first time in years are giving me an idea of what an expression is actually doing. Just because the visualization is kept in a form that is easy to understand with a programming background but can also be translated to the expression itself in a straightforward manner.
Graphs for these really hammer home the point that regular expressions aren't magic. Parsers have so many abilities that when starting out, my expressions were horribly inefficient and missed many corner cases. Learning to graph them just like automata immediately made things easier.
When green devs are having trouble with regular expressions (and don't have a formal computer science background), I like to give them a crash course in DFAs.
I love regex and have no trouble reading them, but still love this tool, great job. I especially like the railroad diagrams, for those cases where I brainfarted on a regex and it's doing something other than what I intended. Thanks for this.
Plug for Verbal Expressions (no affiliation), which has an alternate way of compiling more human-readable regexes for a dozen languages: http://verbalexpressions.github.io/
I remember that library. A year after I made regexpbuilder https://www.npmjs.com/package/regexpbuilder that library suddenly appeared, and was basically a rip-off of the concept I appear to have created (there was no such other library before regexpbuilder), but is also fairly useless because it doesn't look like it could represent more than about 10% of the possible regular expressions. Yet there was no mention of my library at all in the readme of verbal expressions.
Even that is wrong because you can have privately owned TLDs (I forget what they're technically called) like .google
So sundar.pichai@google is technically a valid address (whether .google has any MX records is another matter)
Regex shouldn't really be used for email addresses anyway because the only reliable way to authenticate an email address is to literally send an email to that address.
TLDs can be managed very differently depending on the owners so I don't think it's safe to assume "applies to all" just because there is a rule in place for some gTLDs.
I use regex a lot but deliberately keep it simple.
One thing that confounded me often was positive and negative look-arounds. I always got the expressions mixed up, until I just put the expressions into a table like this...
It's not hard, but for whatever reason my brain had trouble remembering the usage because every time I looked it up, each of those expressions was nested in a paragraph of explanation, and I could not see the simple intuitive pattern.
Putting it into a simple visualization helps a lot.
Now, if I can find a similar mnemonic for backreferences !?
Maybe it's easier to remember that lookbehinds are evil from an implementation standpoint, and even in Perl have arbitrary limitations. If you see lookbehinds, look away! If you see lookaheads, go ahead.
It's something I really like about .NET's regular expressions. Lookbehind has no limitations and will just match backwards with all features you can use in other parts.
So depending on the language or flavor you're working in, running away isn't really necessary.
Oddly, lookbehinds are evil only in a specific backtracking world. We never got around to implementing arbitrary lookarounds in Hyperscan (https://github.com/intel/hyperscan) but if we had done something in the automata world to handle lookaround, lookbehinds are way easier than lookaheads.
To handle a lookbehind, you really only need to occasionally 'AND' together some states (not an operation you would normally do in a standard NFA whether Glushkov or Thompson). To handle lookaheads... well, it gets ugly.
That is not correct. 15 is total maximum number of repeats including the first one. Even the diagram on https://ihateregex.io/expr/username correctly says that loop can be taken between 2 and 14 times.
We use it on slack and irc for debugging people's regular expressions all the time. Being able to have 30 revisions to a base regex to troubleshoot is fantastic.
Is there a bug? In regexp for IPv4: https://ihateregex.io/expr/ip expression ends with {3} but the diagram states "2 times" in lower right - shouldn't it say "3 times"?
> To sum up: RegEx's are misnamed. I think it's a shame, but it won't change. Compatible 'RegEx' engines are not allowed to reject non-regular languages. They therefore cannot be implemented correctly with only Finte State Machines. The powerful concepts around computational classes do not apply. Use of RegEx's does not ensure O(n) execution time. The advantages of RegEx's are terse syntax and the implied domain of character recognition. To me, this is a slow moving train wreck, impossible to look away, but with horrible consequences unfolding
Nope. <h1 class="foo>bar">My First Heading</h1> will misparse. (This is valid HTML 5.) You really need recursive regex or something equivalent in power, otherwise you will always fail.
I've had to write regex for deeply proprietary SQL-like (the word "like" is a big BIG stretch) language. This really is nothing. The regex itself was 4 pages long. AFAIK they still use it in production, almost 10 years later with 0 modifications.
neat site! clicking an example opens up a playground with live update and explanation and railroad diagrams, similar to sites like regex101[1] and regulex[2]
one suggestion would be to mention clearly which tool/language is being used, regex has no unified standard.. based on "Cheatsheet adapted" message at the bottom, I think it is for JavaScript. I wrote a book on js regexp last year, and I have post for cheatsheet too [3]
Very cool! The site that worked best for me to learn regex was https://regexcrossword.com/ - after solving my way through all of them (I got really hooked when I discovered the site) I found I was alright at regex.
OK, these kinds of regex tools get posted quite often. I get it, regex is very confusing at first. And some of these use-cases result in rather complex expressions nobody should be forced to write from scratch (you are still remembering to write unit tests for them though, right?)
But as someone who actually knows [some flavours of] regex fairly well, what I would really like, is a reference that covers all the subtle differences between the various regex engines, along with community-managed documentation (perhaps wiki pages) of which applications & API versions use which flavour of regex.
For example, the other day I wanted to run a find on my NAS. I needed to use a regex, but the Busybox version of find doesn't support the iregex option, so all expressions are case-sensitive. With some googling, I was able to find out that the default regex type is Emacs, but I wasn't able to find either a good reference for exactly what Emacs regex does and doesn't support, nor any information about how to set the "i" flag. In the end I had to manually convert every character into a class (like [aA] for "a") which was tedious, but quicker than trying to find a better solution or resorting to grep.
A related, annoyingly common pattern is that the documentation for `find` states that `--regex` specifies a regex, but it does not state which flavour of regex. The documentation for certain versions of `find`, which support alternative engines, note that the default is Emacs. From this I was able to infer (perhaps wrongly) that the Busybox `find` uses Emacs-flavoured regex, but ultimate I still had to resort to some trial-and-error. This problem is all too common in API documentation.
You're totally right. Right now this tool only supports the javascript flavor of regex. That said, for all the simple expressions shown there it's more or less the same for most other engines. I guess that makes it okay.
The O’Riley book “mastering regular expressions” has a whole section dedicated to it. As well as several tables. But it would be nice to have an online version.
And it's one one the best O'Reilly books. I went and checked because of your comment and just noticed there was a third edition that I missed, I have the second. Still a book worth studying.
Honestly, as a noob, this is one of the biggest reasons I have such a hard time deciding to learn regex.
Python flavor would probably be different than PCRE, which is probably different than JS flavor.
Even worse is that it might be too late to standardize all the regex flavors because there is already so much written in different regex flavors that it just costs too much for them to become obsolete in the future.
Honestly don't let this get you down, here's a learning plan (use regex101 to learn)
1) Learn PCRE regex.
2) Try regex golf or cross words to learn PCRE regex.
3) Take the quiz on regex101.
Once you're done with all 3:
Learn the minor/major differences in the other languages. There aren't many. For example this named capture group:
(?<somename>someregex)
Would look like this in a different language:
(?P<somename>someregex)
There's some differences about what language can and cannot do like recursion because someone thought it was a great idea to make javascript awful at regex, but that's besides the point. Regex is totally worth learning.
The basic regex is easy, infact an English word is a regex! A dot matches a single character. Star multiple of the previous character. Just that is useful for a lot of cases!
The cheat sheets exist because people aren’t learning regex. You don’t need to learn every flavor of regex, just the one or small number you need to know. And once you know the basics, the differences are very minor.
If you believe it is possible to become an expert in regular expressions as they exist in modern computer languages in "a couple of hours at best" you are delusional.
I tend to go to https://www.regular-expressions.info when I need to find out which features are supported between dialects. Not always up-to-date, but has some good info.
RE2 syntax[1] is a pretty good option to learn, because it's mostly a "lowest common denominator" - if it works in RE2, it should work in PCRE, Python, Javascript, etc. The reverse isn't true - there is a bunch of syntax that RE2 doesn't support by design, often to constrain performance bounds.
Emacs regexps are unfortunately their own weird beast - they handle parentheses differently than other regexp engines, because Emacs assumes that you'll be running regexps on Lisp code a lot and want to easily match parentheses. The best documentation on that syntax is (confusingly) in the Elisp reference manual: https://www.gnu.org/software/emacs/manual/html_node/elisp/Sy....
IME Emacs provides a very pleasant way to write regexps using the rx library. ELPA also has the package xr, which converts Elisp regexps to rx format, and pcre2el converts PCRE to Elisp. So a regexp like
Agreed that rx is nice, but really only useful if you're writing elisp. 90%+ of people who need to interact with Emacs regexps aren't writing an elisp program - they're using Emacs interactively, or even using another program (busybox, GNU find, etc.) that uses Emacs regexps for historical reasons. For those people the differences in syntax between Emacs regexps and "normal" regex dialects are a pain.
if you're on osx, the app Patterns is really good for testing regex, and also has quick references for a variety of regex 'engines' and also has decent matching explanations
Its like SQL - everyone has a dialect. For most things where a SQL/regex engine/parser isn't the core of what they do, it will never be a priority. The best approach IMO is something like this in priority order:
1. Stick to using the lowest common denominator like you did for case insensitivity.
2. If that becomes too cumbersome, then consider whether regex is the right tool for the job. Maybe you can use e.g Python/your favorite language with a known regex standard.
3. If there are no other tools and you're stuck with whatever flavor of regex one particular thing supports, only then invest time in learning the details. There is probably a book out there with the details even if there's no webpage.
It's not so bad going between JS, Ruby and Elixir regex (possibly due to my use of a smaller set of features), but VIM regex disappoint me time after time.
One thing i've always missed from the Perl programming language is the regex operators.
You could do:
my $var='foo foo bar and more bar foo!!!';
if($var=~/(foo|bar)/g){ # does the variable contain foo or bar?
print "foo! $1 removing foo..\n";
# remove our value..
$var=~s/$1//g;
}
134 comments
[ 3.0 ms ] story [ 205 ms ] threadThis tool is a cheat sheet that also explains the commonly used expressions so that you understand it.
- There is a visual representation of the regular expression (thanks to regexpr)
- The application shows matching strings which you can play around
- Expressions can be edited and these are instantly validated
When green devs are having trouble with regular expressions (and don't have a formal computer science background), I like to give them a crash course in DFAs.
https://www.benf.org/other/regexview/
2 points:
1. it fiddled with my back button which is a bit annoying
2. a better email sample is
which removes the 2 ampersands problem.I think I know what's wrong with your back button. I will fix it.
And for the regex. will try it out and see if I can add it.
So sundar.pichai@google is technically a valid address (whether .google has any MX records is another matter)
Regex shouldn't really be used for email addresses anyway because the only reliable way to authenticate an email address is to literally send an email to that address.
i.e. johndoe@com will never exist
One thing that confounded me often was positive and negative look-arounds. I always got the expressions mixed up, until I just put the expressions into a table like this...
It's not hard, but for whatever reason my brain had trouble remembering the usage because every time I looked it up, each of those expressions was nested in a paragraph of explanation, and I could not see the simple intuitive pattern.Putting it into a simple visualization helps a lot.
Now, if I can find a similar mnemonic for backreferences !?
So depending on the language or flavor you're working in, running away isn't really necessary.
To handle a lookbehind, you really only need to occasionally 'AND' together some states (not an operation you would normally do in a standard NFA whether Glushkov or Thompson). To handle lookaheads... well, it gets ugly.
The username reference doesn't match 16 characters as claimed
looks good to me
It's how I learned regex years ago, and I still use it today to test/build more complex patterns.
Enjoy: https://news.ycombinator.com/item?id=20614847
Plus the quiz is awesome.
[0] - https://regex101.com/debugger
Because if I specify x{0,3}, i have 2 paths - around x and thru x + at most 2 more times
grabs popcorn
Your sanity won't be left intact tho.
[1] http://www.drregex.com/2018/11/how-to-match-b-c-where-abc-be...
Relies on ?(DEFINE): http://p3rl.org/perlre#(DEFINE)
> To sum up: RegEx's are misnamed. I think it's a shame, but it won't change. Compatible 'RegEx' engines are not allowed to reject non-regular languages. They therefore cannot be implemented correctly with only Finte State Machines. The powerful concepts around computational classes do not apply. Use of RegEx's does not ensure O(n) execution time. The advantages of RegEx's are terse syntax and the implied domain of character recognition. To me, this is a slow moving train wreck, impossible to look away, but with horrible consequences unfolding
¯\_(ツ)_/¯
Unlike most regex helpers, in this one you would start with the text you want to filter/parse and then it would suggest you possible extractions.
Do you know any alternatives?
one suggestion would be to mention clearly which tool/language is being used, regex has no unified standard.. based on "Cheatsheet adapted" message at the bottom, I think it is for JavaScript. I wrote a book on js regexp last year, and I have post for cheatsheet too [3]
[1] https://regex101.com/
[2] https://jex.im/regulex
[3] https://learnbyexample.github.io/cheatsheet/javascript/javas...
If the only thing that is embedded in that frame was taken entirely from a different project, that project should at least be mentioned in the frame.
in the cheatsheet is false. (https://regexr.com/4tc48)
`.` can match any character except linebreaks (including whitespace)
But as someone who actually knows [some flavours of] regex fairly well, what I would really like, is a reference that covers all the subtle differences between the various regex engines, along with community-managed documentation (perhaps wiki pages) of which applications & API versions use which flavour of regex.
For example, the other day I wanted to run a find on my NAS. I needed to use a regex, but the Busybox version of find doesn't support the iregex option, so all expressions are case-sensitive. With some googling, I was able to find out that the default regex type is Emacs, but I wasn't able to find either a good reference for exactly what Emacs regex does and doesn't support, nor any information about how to set the "i" flag. In the end I had to manually convert every character into a class (like [aA] for "a") which was tedious, but quicker than trying to find a better solution or resorting to grep.
A related, annoyingly common pattern is that the documentation for `find` states that `--regex` specifies a regex, but it does not state which flavour of regex. The documentation for certain versions of `find`, which support alternative engines, note that the default is Emacs. From this I was able to infer (perhaps wrongly) that the Busybox `find` uses Emacs-flavoured regex, but ultimate I still had to resort to some trial-and-error. This problem is all too common in API documentation.
Python flavor would probably be different than PCRE, which is probably different than JS flavor.
Even worse is that it might be too late to standardize all the regex flavors because there is already so much written in different regex flavors that it just costs too much for them to become obsolete in the future.
This is really demotivating.
1) Learn PCRE regex. 2) Try regex golf or cross words to learn PCRE regex. 3) Take the quiz on regex101.
Once you're done with all 3:
Learn the minor/major differences in the other languages. There aren't many. For example this named capture group:
(?<somename>someregex)
Would look like this in a different language:
(?P<somename>someregex)
There's some differences about what language can and cannot do like recursion because someone thought it was a great idea to make javascript awful at regex, but that's besides the point. Regex is totally worth learning.
Clear your afternoon, and just learn it. Seriously, it takes a couple of hours at best and then - BOOM - you're done for the rest of your life.
If that were so easy then I don't think much of these cheatsheets would exist.
Emacs regexps are unfortunately their own weird beast - they handle parentheses differently than other regexp engines, because Emacs assumes that you'll be running regexps on Lisp code a lot and want to easily match parentheses. The best documentation on that syntax is (confusingly) in the Elisp reference manual: https://www.gnu.org/software/emacs/manual/html_node/elisp/Sy....
[1] https://github.com/google/re2/wiki/Syntax
https://krillapps.com/patterns/
https://rexegg.com/regex-quickstart.html
1. Stick to using the lowest common denominator like you did for case insensitivity.
2. If that becomes too cumbersome, then consider whether regex is the right tool for the job. Maybe you can use e.g Python/your favorite language with a known regex standard.
3. If there are no other tools and you're stuck with whatever flavor of regex one particular thing supports, only then invest time in learning the details. There is probably a book out there with the details even if there's no webpage.
Then pray you never get to step 3 :)
It's not so bad going between JS, Ruby and Elixir regex (possibly due to my use of a smaller set of features), but VIM regex disappoint me time after time.
You could do: