However, I'd suggest to reorganize the chapters so that features not yet introduced aren't shown in examples without explanations. For example, you explain anchors and quantifiers many chapters later but use them liberally in earlier chapters without explaining them.
I wouldn't worry too much about making the examples trivial. That just makes it easy to learn! There are probably lot's of good orders, but I'd probably go something like:
IMO character classes are quite an advanced feature (or at least confusing for beginners) because of being character orientated. They also don't tend to very useful unless you've already covered repetition.
Yes - in the Character classes chapter you had just introduced the negate operator, then in the next example use the beginning operator, which happens to be the same character in a different context. That might be a leap too far. Great resource!
Railroad diagrams may help. I use a three step approach to present one example of recursion, which includes showing the difference between two-level nesting vs recursive version [1]. This is the railroad visualization for two-level [2].
I like the example based approach. I learn from examples far quicker than I learn from “explanations”. If I attempt to learn from an example and my brain hits an exception, only then do I start reading the supporting text.
Nice approach. You’ve made a valuable thing and implemented a powerful idea.
I've been using tldr [1] instead of man pages lately to get started with a command (or to remind myself how to use one). I've learned a lot just by reading the examples shown, and then read the man pages if I am missing something.
Examples being greater than explanations is one of the main reasons I emphasize explanative error messaging, clear simple typings, and verbose function/variable names over documentation.
Docs are really good for discovery and should cover many topics shallowly so you can glean a big picture quickly. I generally don't like going to them for specs that could have just been an error message, a type, or a better naming convention.
Constructive criticism: I was about to send this to a friend who is new to programming, but the introduction is just too short. It would be great if the introduction included one or two motivational examples for the types of trouble you run into when you don't know regular expressions.
One thing not mentioned here which I think is good to be aware of as you write intermediate to advanced regexes is understanding "catastrophic backtracking" and how to mitigate it: https://www.regular-expressions.info/catastrophic.html
For some reason I enjoy figuring regexes out. What I usually do is TDD them, I have a mini test suite of examples of strings I want to match and strings I don't want to match and I write some code to apply a candidate regex to them all and validate, and then I iterate until it passes. Then I rewrite the regex in extended regex format and add comments so that other people or future me understand what's going on.
Doing what a good regex can do with regular code instead (which you might do with the goal of readability or maintainability) is usually much much MUCH slower, FYI
I started teaching Python to my GF(she is working from home and now has plenty of time to do some extra learning). She is not a programmer and I have been giving her small functions to write. Recently, we started with RegEx and she finds it really hard to get into. She wants to see examples and follow along. I think this will be perfect for her and anyone starting out to learn regex.
Lots of praise here, so I won't re-iterate the good points (presentation, pleasant tone, good structuring) and will head straight to the meat of my issue with the title:
This is not a book for regular folk.
A regular HN reader, sure. A technically inclined interested party who wants to break the ice with Regexes, sure. But not regular folk.
Here is what I'm talking about:
> Introduction
> Regular expressions (“regexes”) allow defining a pattern
Ok, with you so far. As a layman, though, I would be very much be looking for you to expand on what you mean by 'pattern'.
> and executing it against strings.
"Executing" gets a wrinkled brow. "Strings" gets a squinty eye. "executing against strings" and you've lost me. There's now too much new information in this sentence for me to be on board with it. If I knew what all those terms meant and the context with which they are meaningful, I probably wouldn't be trying to read 'RegEx for Regular Folk'.
> Substrings which match the pattern are termed “matches”.
As above, but it's also slightly confusing here that we're defining matches and we haven't even talked about what a pattern is yet. As such, I can't even visualise or conceptualise what I would be matching or similar. If I press on regardless, this is just some unresolved debt that I will have to reconcile later or I will just get frustrated and put the book down.
> A regular expression is a sequence of characters that define a search pattern.
Ah, good, we're defining a pattern after we've already described a 'match'.
> Regex finds utility in:
>input validation
And straight out the bat we're hit with a term that is only going to be relevant for techie people. Unless you are aiming this at techie people. But aren't we aiming this at 'regular folk'?
The above is really just my long drawn out beef with 'x for the masses', 'y for mere mortals' and the like. For me the best explanation of regular expressions comes from Al Sweigart in 'Automate the Boring Stuff with Python' [1]. He not only gives a pretty thorough explanation of pattern matching before bringing in any domain-specific terms, but he also motivates why you would want to pattern match in the first place. He gives context for circumstances under which you might reach for regex as a tool.
I'm looking through the later pages of this book and as a techperson I'm thinking 'this is beautiful. I can see the examples clearly, there is a clear correlation between the visuals and the exercise.' I'm also thinking as a folk person 'when the hell will I need a match? Under what circumstances and I going to need to know that there is one 'p' in 'grape' but two 'p's in 'apple'? What use is writing a pattern to match against certain fruits and utility items?
So yes, basically, after all that I can summarise "good book, bad title".
Just some 2cent feedback - don't assume anything is known.
The BASIC lesson doesn't mention anything about /g. Having not touched regex in years I had no idea what that was and kept thinking 'why isn't he showing it matching a g if he has that in the example'.
RegExr [0] does a great job of showing individual highlights even when they are in a sequential string. You can try to implement this if you want instead of showing a callout with a note to let the reader know that they highlights should be on individual characters.
o ^ character outside brackets
o $ end of line
o +
But the explanation above does not introduce these yet, so a real beginner user (like me) is lost. The ambigious characters example is fine, since it uses all the concepts already explained.
This looks like a great resource! Like others, I vastly prefer an example-based style, and the examples are really well chosen and very illustrative. I generally think I know my regexes, but I've already learned a few tricks. (Backreferences to match different delimiter options but not mixed delimiters is very cool!)
Feedback:
The highlighting of matches is slightly shifted to the left for me in Firefox 75 but not in Chrome (both on Ubuntu 16.04). The shift is subtle but enough to make me have to look two or three times at most examples, as the highlight covers half of the character before the match and only half of the last character in the match. Can I suggest adding Firefox to your test regimen, if you haven't already? :)
Also, on the Anchors page, I believe "carat" should be spelled "caret."
Thanks for this once again! I will definitely be revisiting this site to brush up and learn new tricks. Especially lookaround, which I have never quite wrapped my head around!
> The highlighting of matches is slightly shifted to the left … Can I suggest adding Firefox to your test regimen, if you haven't already? :)
Oh, I thought I had fixed that. I primarily test with Firefox, so this is a bit of a surprise. I'll check it out—I think it's something to do with CSS's `letter-spacing`.
I personally had the most trouble with regexes because I didn't have a good mental model of how they worked. The hard part wasn't finding the correct symbol/character class I was trying to match, but coming to grips with repetition, greedy/nongreedy, etc.
I took a compilers class in college where one of the projects was to implement a simple regex matcher using NFAs. Bashing my head against this for a week really helped with being able to "read" a regex. Not sure if this was due to finally understanding the algorithm, or the fact that I was just constantly staring at broken regex matches all day.
IMO it was a fairly small time investment for something that is so widely used.
I'm not sure how to explain it but the most important thing I've learned in over a decade of programming experience is to not use regular expressions for many things they may seem like a regular expression problem.
For example, even something as simple a phone number can have all sorts of weird but valid variants. Be sure you really need to even validate it's format and not just that it's present.
Trying to handle all of those variants via regex expression is doable but a pain. And in practice you as the programmer should not be defining those variants that are valid as it's up to the business itself to define what type of data it considers to be valid for the field.
That said I've also worked for companies with small engineering teams where the goal has always been to be as efficient with development time as possible, as opposed to making a near ideal system. Software has different needs when it's used by a thousand people than when it's used by millions.
I think it is understanding the algorithm that does it.
I also recommend that people learn how to read a regex by writing a small recursive program to match specific regexes. After you look at a regex and think about how it might work, intuition follows.
Actually writing the bit that turns the regex expression into said program isn't as important though. Doing that by hand 5 times is enough IMO.
112 comments
[ 3.6 ms ] story [ 176 ms ] thread:)
However, I'd suggest to reorganize the chapters so that features not yet introduced aren't shown in examples without explanations. For example, you explain anchors and quantifiers many chapters later but use them liberally in earlier chapters without explaining them.
I'll work on making things clearer.
- Literal strings - Optional characters - Optional strings of characters (using groups) - Alternations (using groups) - Repetitions (using groups)
Then move onto to things like character classes.
IMO character classes are quite an advanced feature (or at least confusing for beginners) because of being character orientated. They also don't tend to very useful unless you've already covered repetition.
https://www.rexegg.com/regex-recursion.html
[1] https://github.com/learnbyexample/py_regular_expressions/blo...
[2] https://www.debuggex.com/r/SMLRfiyt0Ag2hXu5
Nice approach. You’ve made a valuable thing and implemented a powerful idea.
[0] https://www.attrs.org/en/stable/examples.html
[1] https://github.com/tldr-pages/tldr
Docs are really good for discovery and should cover many topics shallowly so you can glean a big picture quickly. I generally don't like going to them for specs that could have just been an error message, a type, or a better naming convention.
For some reason I enjoy figuring regexes out. What I usually do is TDD them, I have a mini test suite of examples of strings I want to match and strings I don't want to match and I write some code to apply a candidate regex to them all and validate, and then I iterate until it passes. Then I rewrite the regex in extended regex format and add comments so that other people or future me understand what's going on.
Doing what a good regex can do with regular code instead (which you might do with the goal of readability or maintainability) is usually much much MUCH slower, FYI
One visual enhancement that could be really helpful would be to hover over the regex or the match and see the reciprocal highlighted.
This is not a book for regular folk.
A regular HN reader, sure. A technically inclined interested party who wants to break the ice with Regexes, sure. But not regular folk.
Here is what I'm talking about:
> Introduction
> Regular expressions (“regexes”) allow defining a pattern
Ok, with you so far. As a layman, though, I would be very much be looking for you to expand on what you mean by 'pattern'.
> and executing it against strings.
"Executing" gets a wrinkled brow. "Strings" gets a squinty eye. "executing against strings" and you've lost me. There's now too much new information in this sentence for me to be on board with it. If I knew what all those terms meant and the context with which they are meaningful, I probably wouldn't be trying to read 'RegEx for Regular Folk'.
> Substrings which match the pattern are termed “matches”.
As above, but it's also slightly confusing here that we're defining matches and we haven't even talked about what a pattern is yet. As such, I can't even visualise or conceptualise what I would be matching or similar. If I press on regardless, this is just some unresolved debt that I will have to reconcile later or I will just get frustrated and put the book down.
> A regular expression is a sequence of characters that define a search pattern.
Ah, good, we're defining a pattern after we've already described a 'match'.
> Regex finds utility in:
>input validation
And straight out the bat we're hit with a term that is only going to be relevant for techie people. Unless you are aiming this at techie people. But aren't we aiming this at 'regular folk'?
The above is really just my long drawn out beef with 'x for the masses', 'y for mere mortals' and the like. For me the best explanation of regular expressions comes from Al Sweigart in 'Automate the Boring Stuff with Python' [1]. He not only gives a pretty thorough explanation of pattern matching before bringing in any domain-specific terms, but he also motivates why you would want to pattern match in the first place. He gives context for circumstances under which you might reach for regex as a tool.
I'm looking through the later pages of this book and as a techperson I'm thinking 'this is beautiful. I can see the examples clearly, there is a clear correlation between the visuals and the exercise.' I'm also thinking as a folk person 'when the hell will I need a match? Under what circumstances and I going to need to know that there is one 'p' in 'grape' but two 'p's in 'apple'? What use is writing a pattern to match against certain fruits and utility items?
So yes, basically, after all that I can summarise "good book, bad title".
[1] https://automatetheboringstuff.com
I'll try easing the curve, especially early on and make clearer the intended audience.
More regex resources I rely on:
http://www.regexr.com/
https://gchq.github.io/CyberChef
https://regexper.com/#.%3F%5Bv%2Ci%5D.*
https://cheatography.com/davechild/cheat-sheets/regular-expr...
Also, those are some amazing resources, especially CyberChef.
The BASIC lesson doesn't mention anything about /g. Having not touched regex in years I had no idea what that was and kept thinking 'why isn't he showing it matching a g if he has that in the example'.
[0] https://regexr.com/
Feedback:
- In the chapter https://refrf.shreyasminocha.me/chapters/character-classes an example is given which uses:
But the explanation above does not introduce these yet, so a real beginner user (like me) is lost. The ambigious characters example is fine, since it uses all the concepts already explained.Feedback:
The highlighting of matches is slightly shifted to the left for me in Firefox 75 but not in Chrome (both on Ubuntu 16.04). The shift is subtle but enough to make me have to look two or three times at most examples, as the highlight covers half of the character before the match and only half of the last character in the match. Can I suggest adding Firefox to your test regimen, if you haven't already? :)
Also, on the Anchors page, I believe "carat" should be spelled "caret."
Thanks for this once again! I will definitely be revisiting this site to brush up and learn new tricks. Especially lookaround, which I have never quite wrapped my head around!
Oh, I thought I had fixed that. I primarily test with Firefox, so this is a bit of a surprise. I'll check it out—I think it's something to do with CSS's `letter-spacing`.
I've fixed the typo, thanks for pointing it out.
Thanks for the comments!
I took a compilers class in college where one of the projects was to implement a simple regex matcher using NFAs. Bashing my head against this for a week really helped with being able to "read" a regex. Not sure if this was due to finally understanding the algorithm, or the fact that I was just constantly staring at broken regex matches all day.
IMO it was a fairly small time investment for something that is so widely used.
I'll recommend this post that's been on HN many times: https://swtch.com/~rsc/regexp/regexp1.html
For example, even something as simple a phone number can have all sorts of weird but valid variants. Be sure you really need to even validate it's format and not just that it's present.
Trying to handle all of those variants via regex expression is doable but a pain. And in practice you as the programmer should not be defining those variants that are valid as it's up to the business itself to define what type of data it considers to be valid for the field.
That said I've also worked for companies with small engineering teams where the goal has always been to be as efficient with development time as possible, as opposed to making a near ideal system. Software has different needs when it's used by a thousand people than when it's used by millions.
I also recommend that people learn how to read a regex by writing a small recursive program to match specific regexes. After you look at a regex and think about how it might work, intuition follows.
Actually writing the bit that turns the regex expression into said program isn't as important though. Doing that by hand 5 times is enough IMO.