The parallel is apt, but regex /o is more like a closure that captures by value at declaration time rather than an ambiguity between capture strategies.
This is one of the features that Ruby cribbed directly from Perl. The Ruby documentation seems really bad, in particular “interpolation mode” is grievously misleading.
Perl’s documentation is far more clear about the consequences:
o Compile pattern only once.
[…]
PATTERN may contain variables, which will be
interpolated every time the pattern search is
evaluated, except for when the delimiter is a
single quote. […] Perl will not recompile the
pattern unless an interpolated variable that
it contains changes. You can force Perl to skip
the test and never recompile by adding a /o
(which stands for "once") after the trailing
delimiter. Once upon a time, Perl would recompile
regular expressions unnecessarily, and this
modifier was useful to tell it not to do so,
in the interests of speed. But now, the only
reasons to use /o are one of:
[reasons]
The bottom line is that using /o is almost
never a good idea.
In the days before Perl automatically memoized the compilation of regexes with interpolation, even back in the 1990s, it said,
However, mentioning /o constitutes a promise
that you won't change the variables in the
pattern. If you change them, Perl won't even
notice.
PATTERN may contain references to scalar
variables, which will be interpolated
(and the pattern recompiled) every time the
pattern search is evaluated. […] If you want
such a pattern to be compiled only once, add
an “o” after the trailing delimiter. This
avoids expensive run-time recompilations, and
is useful when the value you are interpolating
won't change over the life of the script.
Nowadays, computers are super fast and super huge and super always available.
But even in the early 2000s, that was not the case. I know this b/c one of my first jobs out of college was to build a regex based system to analyze spam emails sent to big providers (like Yahoo and Hotmail (Microsoft)".
IIRC, we only had ONE Dell 2600 box to do all of the above with literally millions of emails and the same box did the parsing and the database storage (via MySql).
You REALLY learn what makes regexes efficient versus not and I remember reading about "/o" and testing it to see if it made a difference for what I was doing. I don't remember either way what the results were but this was def a blast from the past.
> I didn’t recognize /o. It didn’t seem critically important to lookup yet.
> With nothing else to investigate, I finally looked up the docs for what the /o regex modifier does.
I'll probably never understand this mode of thinkning. But then again, Ruby programmers are, after all, people who chose to write Ruby.
> /o is referred to as “Interpolation mode”, which sounded pretty harmless.
Really? Those words sound quite alarming to me, due to personal reminiscences of eval.
Also, this whole "/o" feaure seems insane. If I have an interpolation in my regex, obviously I have to re-interpolate it every time a new value is submitted, or I'd hit this very bug. And if the value is expected to the same every time, then I can just compile it once and save the result myself, right? In which case, I probably could even do without interpolation in the first place.
It's a feature dating from the 1990s, when Perl (and I guess Ruby?) didn't have a way for the user to store a compiled regex, and this was a useful shortcut for a very specific optimization, which Ruby documented badly. Perl (and I guess Ruby?) later evolved in a way that made /o unnecessary, but the (now mis)feature remained.
> Modifier o means that the first time a literal regexp with interpolations is encountered, the generated Regexp object is saved and used for all future evaluations of that literal regexp.
That is crystal clear to me. It means that on the next execution, the new values of the interpolation will be ignored; the regexp is now "baked" with the first ones.
Like this in C++:
void fun(int arg)
{
static int once = arg;
}
if we call this as f(42) the first time, once gets initialized to 42. If we then call it f(73), once stays 42.
There is a function in POSIX for once-only initializations: pthread_once. C++ compilers for multithreaded environments emit thread-safe code to do something similar to pthread_once to ensure that even if there are several concurrent first invocations of the function, the initialization happens once.
This is a footgun. A language should strive not to add footguns. Every footgun you provide, somebody is going to blow their foot off with it, so that's a high price. If your language is popular it might be a lot of somebodies.
The opposite behaviour (we have a constant regular expression, we re-use it often but the tooling doesn't realise and so it's created each time we mention it) is not a footgun, it results in poor performance, and so you might want (especially in some managed languages) to just magically optimise this case, but if not you won't cause mysterious bugs. An expert, asked "Why is this slow?" can just fix it - you have to supply basic tools for that, but this flag is not a sensible tool.
18 comments
[ 3.1 ms ] story [ 39.5 ms ] thread"Some people, when confronted with a problem, think 'I know, I'll use regular expressions.' Now they have two problems."
Perl’s documentation is far more clear about the consequences:
(https://perldoc.perl.org/perlop#Regexp-Quote-Like-Operators)
In the days before Perl automatically memoized the compilation of regexes with interpolation, even back in the 1990s, it said, Perl 4’s documentation is briefer. It says,(https://github.com/Perl/perl5/blob/perl-4.0.00/perl.man#L272...)
But even in the early 2000s, that was not the case. I know this b/c one of my first jobs out of college was to build a regex based system to analyze spam emails sent to big providers (like Yahoo and Hotmail (Microsoft)".
IIRC, we only had ONE Dell 2600 box to do all of the above with literally millions of emails and the same box did the parsing and the database storage (via MySql).
You REALLY learn what makes regexes efficient versus not and I remember reading about "/o" and testing it to see if it made a difference for what I was doing. I don't remember either way what the results were but this was def a blast from the past.
PS If you want to read more about the spam work back the, I have a Twitter thread about it here: https://x.com/alexpotato/status/1208948480867127296
> With nothing else to investigate, I finally looked up the docs for what the /o regex modifier does.
I'll probably never understand this mode of thinkning. But then again, Ruby programmers are, after all, people who chose to write Ruby.
> /o is referred to as “Interpolation mode”, which sounded pretty harmless.
Really? Those words sound quite alarming to me, due to personal reminiscences of eval.
Also, this whole "/o" feaure seems insane. If I have an interpolation in my regex, obviously I have to re-interpolate it every time a new value is submitted, or I'd hit this very bug. And if the value is expected to the same every time, then I can just compile it once and save the result myself, right? In which case, I probably could even do without interpolation in the first place.
That is crystal clear to me. It means that on the next execution, the new values of the interpolation will be ignored; the regexp is now "baked" with the first ones.
Like this in C++:
if we call this as f(42) the first time, once gets initialized to 42. If we then call it f(73), once stays 42.There is a function in POSIX for once-only initializations: pthread_once. C++ compilers for multithreaded environments emit thread-safe code to do something similar to pthread_once to ensure that even if there are several concurrent first invocations of the function, the initialization happens once.
The opposite behaviour (we have a constant regular expression, we re-use it often but the tooling doesn't realise and so it's created each time we mention it) is not a footgun, it results in poor performance, and so you might want (especially in some managed languages) to just magically optimise this case, but if not you won't cause mysterious bugs. An expert, asked "Why is this slow?" can just fix it - you have to supply basic tools for that, but this flag is not a sensible tool.