TIL that Ruby has mutable strings, and (until the announced change) even had them mutable by default (and the change only affects literal strings; non-literal strings are still mutable). Python has always only ever had immutable strings.
Strings will still be mutable by default after the change which only makes string literals always frozen (which has been a file-level opt-in for a while.)
Is it the future path of any successful JIT / dynamic typed / scripting language to realize they needed all optimizations from compiled / statically typed / lower level languages ?
Would Ruby be as successful if they had all those complicated features right from the start ?
Or do all languages start from a nice simple clean slate tabula rasa to get developers hooked, until the language is enough famous to get well developed and starts to be similar to all others big programming languages ?
Even if you have an incompatible codebase that you don't wish to convert, you'll be able to set `RUBYOPT="--disable-frozen-string-literal"` so it keeps running.
And since that flag really doesn't require lots of work in the VM, it's likely to be kept around pretty much forever.
This looks like a really thoughtful and well-planned transition, with years of opt-in warnings followed by years of opt-out warnings before the actual cutover. I’m constantly impressed with the methodical approach of the Ruby team to constantly improving the language in ways that disrupt their users as little as possible.
Btw, OCaml also transitioned from read-write Strings to read-only Strings, and Buffer to be that read-write string. It was introduced in 4.02 released September 2014.
I recall it was a bit bumpy, but not all that rough in the end. I suppose static type checking helps here to find all the ways how it could be used. There was a switch to allow running old code (to make strings and buffers interchangeable).
> Btw, OCaml also transitioned from read-write Strings to read-only Strings
Ruby is not doing that, it's transitioning from mutable strings that can be frozen with no special treatment of literals (unless you opt-in to literals being frozen on per file basis) to mutable strings with all string literals frozen.
How does this work under the hood? Does Ruby keep a giant map of all strings in the application to check new strings against to see if it can dedupe? Does it keep a reference count to each unique string that requires a set lookup to update on each string instance’s deallocation? Set lookups in a giant set can be pretty expensive!
> How does this work under the hood? Does Ruby keep a giant map of all strings in the application to check new strings against to see if it can dedupe?
1. Strings have a flag (FL_FREEZE) that are set when the string is frozen. This is checked whenever a string would be mutated, to prevent it.
2. There is an interned string table for frozen strings.
> Does it keep a reference count to each unique string that requires a set lookup to update on each string instance’s deallocation?
This I am less sure about, I poked around in the implementation for a bit, but I am not sure of this answer. It appears to me that it just deletes it, but that cannot be right, I suspect I'm missing something, I only dig around in Ruby internals once or twice a year :)
I wonder what the basis is for the description of the 3.7 / 4 ruby releases is. I haven't seen this transition plan with version numbers described outside of this blog post.
Well it is not quite a mutable vs immutable strings war, nor Ruby being late to the party or something like that.
The move is so we can avoid allocating a string each we declare and use it since it will be frozen by default. It is a big optimization for GC mainly. Before we had to do such optimization by hand if we intend not to modify it:
# before
def my_method
do_stuff_with("My String") # 1 allocation at each call
end
# before, optim
MY_STRING = "My String".freeze # this does 2 allocations with 1 at init being GC quite early
def my_method
do_stuff_with(MY_STRING)
end
# after
def my_method
do_stuff_with("My String") # 1 allocation first time
end
But this move also complicates strings manipulation in the sense of it will lean users toward immutable ops that tend to allocate a lot of strings.
foo.upcase.reverse
# VS
bar = foo.dup
bar.upcase!
bar.reverse!
So now we have to be deliberate about it:
my_string = +"My String" # it is not frozen
We have frozen string literals for quite a while now, enabled file by file with the "frozen_string_literal: true" comment and I've seen it as the recommended way by the community and the de-facto standard in most codebase I've seen. It is generally enforced by code quality tools like Rubocop.
So the mutable vs immutable is well known, and as it is part of the language, well, people should know the ins and outs.
I'm just a bit surprised that they devised this long path toward real frozen string literals, because it is already ongoing for years with the "frozen_string_literal: true" comment. Maybe to add proper warnings etc. in a way that does not "touch" code ? I prefer the explicit file by file comment. And for deps, well, the version bump of Ruby adding frozen string literals by default is quite a filter already.
It is sorta late to the party. Common Lisp has similar with regards to how lists are done. Specifically, it is not uncommon to make a static list like `'(1 2 3)`. Doing this, however, has implications on what operations you can do on the data elsewhere.
I say sorta late to the party, as I think it is more than fair to say there was not much of a party that folks were interested in in the lisp world. :D
Has anyone actually benchmarked the use of frozen string literals? I feel like this is one of those micro-optimizations that everyone does, but they're probably accomplishing a diminishingly small performance improvement, while making the codebase less readable. On net, a negative.
which, without `# frozen_string_literal: true`, I believe allocates a string when the application loads (it sounds like it might be 2) and another string at runtime and then mutate that.
because that will allocate the frozen string to `FOO` when the application loads, then make a copy of it to `foo` at runtime, then mutate that copy. That means two strings that never leave memory (FOO, SUB_ME) and one that has to be GCed (return value) instead of just one that never leaves memory (SUB_ME) and one that has to be GCed (foo/return value).
This is true in particular when FOO is only used in `my_method`. If it's also used in `my_other_method` and it logically makes sense for both methods to use the same base string, then it's beneficial to use the wider-scope constant.
(The reason this seems reasonable in an application is that the method defines the string, mutates it, and sends it along, which primarily works because I work on a small team. Ostensibly it should send a frozen string, though I rarely do that in practice because my rule is don't mutate a string outside the context in which it was defined, and that seems sensible enough.)
Am I mistaken and/or is there another, perhaps more common pattern that I'm not thinking about that makes this desirable? Presumably I can just add # frozen_string_literal: false to my files if I want so this isn't a complaint. I'm just curious to know the reasoning since it is not obvious to me.
We implemented this recently on a Rails project as part of a Rubocop integration. It actually uncovered a lot of subtle bugs, though I will say that the Ruby language lends itself to buggy code. Thankfully we have sophisticated tooling these days that (mostly) mitigates this.
The phrase “Frozen String Literals” is kind of weird to me. When I assign a string literal to a variable, I do not think of the variable itself as a “string literal.” That phrase is for the literal characters in between quotes in the code, which by definition are already “frozen.” They’re a static part of the code itself. This change makes it so that you cannot mutate a variable which was initialized using a string literal. (if I understand correctly!)
Many comments about Python 2-3 moves. The problem with Python was that 2 to 3 offers little to no incentive.
So I sometimes wonder why JIT isn't used as a motivation to move / remove features. Basically if you want JIT to work, your code has to be x ready or without feature x. So if you still want those performance improvements you will have to move forward.
32 comments
[ 2.3 ms ] story [ 59.5 ms ] threadWould Ruby be as successful if they had all those complicated features right from the start ?
Or do all languages start from a nice simple clean slate tabula rasa to get developers hooked, until the language is enough famous to get well developed and starts to be similar to all others big programming languages ?
An obviously good change, actually massive performance improvements not hard to implement but its still gonna be such a headache and dependency hell
And since that flag really doesn't require lots of work in the VM, it's likely to be kept around pretty much forever.
I recall it was a bit bumpy, but not all that rough in the end. I suppose static type checking helps here to find all the ways how it could be used. There was a switch to allow running old code (to make strings and buffers interchangeable).
Ruby is not doing that, it's transitioning from mutable strings that can be frozen with no special treatment of literals (unless you opt-in to literals being frozen on per file basis) to mutable strings with all string literals frozen.
1. Strings have a flag (FL_FREEZE) that are set when the string is frozen. This is checked whenever a string would be mutated, to prevent it.
2. There is an interned string table for frozen strings.
> Does it keep a reference count to each unique string that requires a set lookup to update on each string instance’s deallocation?
This I am less sure about, I poked around in the implementation for a bit, but I am not sure of this answer. It appears to me that it just deletes it, but that cannot be right, I suspect I'm missing something, I only dig around in Ruby internals once or twice a year :)
But not actually stated it's the plan. I'd bet whatever LLM wrote the article took it as a stronger statement than it is.
The move is so we can avoid allocating a string each we declare and use it since it will be frozen by default. It is a big optimization for GC mainly. Before we had to do such optimization by hand if we intend not to modify it:
But this move also complicates strings manipulation in the sense of it will lean users toward immutable ops that tend to allocate a lot of strings. So now we have to be deliberate about it: We have frozen string literals for quite a while now, enabled file by file with the "frozen_string_literal: true" comment and I've seen it as the recommended way by the community and the de-facto standard in most codebase I've seen. It is generally enforced by code quality tools like Rubocop.So the mutable vs immutable is well known, and as it is part of the language, well, people should know the ins and outs.
I'm just a bit surprised that they devised this long path toward real frozen string literals, because it is already ongoing for years with the "frozen_string_literal: true" comment. Maybe to add proper warnings etc. in a way that does not "touch" code ? I prefer the explicit file by file comment. And for deps, well, the version bump of Ruby adding frozen string literals by default is quite a filter already.
Well, Ruby is well alive and it is what matters)
I say sorta late to the party, as I think it is more than fair to say there was not much of a party that folks were interested in in the lisp world. :D
The original plan was to make the breaking change in 3.0, but that plan was canceled because it broke too much code all at once.
Hence why I proposed this multi-step plan to ease the transition.
See the discussion on the tracker if you are curious: https://bugs.ruby-lang.org/issues/20205
That seems like it's better than doing
because that will allocate the frozen string to `FOO` when the application loads, then make a copy of it to `foo` at runtime, then mutate that copy. That means two strings that never leave memory (FOO, SUB_ME) and one that has to be GCed (return value) instead of just one that never leaves memory (SUB_ME) and one that has to be GCed (foo/return value).This is true in particular when FOO is only used in `my_method`. If it's also used in `my_other_method` and it logically makes sense for both methods to use the same base string, then it's beneficial to use the wider-scope constant.
(The reason this seems reasonable in an application is that the method defines the string, mutates it, and sends it along, which primarily works because I work on a small team. Ostensibly it should send a frozen string, though I rarely do that in practice because my rule is don't mutate a string outside the context in which it was defined, and that seems sensible enough.)
Am I mistaken and/or is there another, perhaps more common pattern that I'm not thinking about that makes this desirable? Presumably I can just add # frozen_string_literal: false to my files if I want so this isn't a complaint. I'm just curious to know the reasoning since it is not obvious to me.
So I sometimes wonder why JIT isn't used as a motivation to move / remove features. Basically if you want JIT to work, your code has to be x ready or without feature x. So if you still want those performance improvements you will have to move forward.