I've been playing with Elixir in my background time; I'm liking it so far. What, if any, effect does a new release of Erlang/OTP have on Elixir? Is there usually a new follow on release of Elixir that chases the new features of the new Erlang release?
A lot of OTP improvements benefit whatever version of Elixir you're using or you can access directly from the erlang library (e.g. When OTP 22 came out, you could use the new socket api (:socket.open(...)).
But some additions require a new Elixir release. One that comes to mind was the new guard clause to check if a key is in a map.
Improvement to the VM or underlying stdlib that are API compatible are "free"
But. Elixir maintain a 2 otp release compatibility. So the elixir stdlib cannot use new functionality until the release of elixir in 2 years.
You can, ofc, just call the erlang lib yourself.
So on this one, you get the JIT or my float_to_string pretty printer speedup for free. But the Map module cannot use the new maps functions for 2 years.
If you use a version of Elixir built against the correct OTP release, you'll get the following automatically:
VM Performance improvements - normally this is minor but for this release it's a big big thing
Bug fixes - to standard library and VM
New Erlang APIs to use directly - you can call Erlang functions easily from Elixir, allowing you to leverage new standard library features, but they won't be used by Elixir because of the compatibility guarantees discussed elsewhere here
The most immediate impact I'm noticing locally is a ~25-30% improvement in compile times from the JIT, which is always a delightful find in any release.
Synthetic benchmarks of the Elixir GraphQL implementation show a 64% improvement! Very excited to see how this plays out with real load, but this is very promising.
As a heavy user of Absinthe in production, I am also very excited to hear that.
I just went to leave a slack message to my team about your report in the middle of the night, and discovered that another teammate had already just excitedly mentioned it. Thanks for what you do!
The EEP dives right into the problem of "late replies" with little detail explaining the concept. I haven't ran into the problem the EEP is trying to solve but I am interested in understanding it. Can anyone provide more details on how a late reply emerges during gen_server client/server message passing and how an alias is intended to solve the issue? Is this only a problem with distributed messages?
It looks like "late replies" are a problem under an additional constraint that's mentioned in the Motivation section,
"When the author of the client code has full control over the client process such late replies can be handled without a proxy since the code can be aware of these potential stray messages and drop them when received."
So.. you have a server sending messages over distributed erlang to a client that the author doesn't control which can risk receiving late messages if a connection is lost? Ugh.. what?
I tried thinking about this from the proxy process angle as a solution to this mysterious problem,
"Since the proxy process is not alive, a reply will be silently dropped and no stray message will reach the previous client of the request."
But, if a "connection lost" event occurs, why would a message make it to the client to begin with? Why is the client still alive? If the connection is re-establish, then why drop the message?
Wish the EEP more clearly articulated the class of bugs / issues its intending to address and when they occur.
If you do gen_X:call and the timeout expires before you receive a response, you may still get a response later (late reply). The timeout case can happen without a connection loss or even without distribution. I believe gen_X:call sets a monitor (maybe a link?) though, so if the server process is distributed, and the connection is lost, the call will fail; however, the server does not set a monitor on the client (to my knowledge), so when it finishes processing the message, it will reply to the client; distribution may have already reconnected, or sending the reply may cause a reconnection. I don't have great knowledge on the monitor, where I worked, we used a modified call that didn't set monitors, because it was too much overhead; we relied on timeouts instead.
If the call was part of a library function, you can return {error, timeout} or whatever, but the process you ran in will have that reply message in its mailbox, possibly forever if it only does selective receives. This will show up in your system as growing memory use, and slowing message processing (unless all of your messaging is selective receives with new references that benefits from the mailbox marking optimization).
In the proxy process alternative, you would spawn a new process to do the gen_X:call, and message back to the real client process. Any late replies get discarded as the spawned process will be dead. But spawning a process isn't free, even if an Erlang process is much lower cost than an OS thread or process. Without looking at details of the implementation, using a process alias seems like a reasonable optimization.
15 comments
[ 2.8 ms ] story [ 30.9 ms ] threadBut some additions require a new Elixir release. One that comes to mind was the new guard clause to check if a key is in a map.
But. Elixir maintain a 2 otp release compatibility. So the elixir stdlib cannot use new functionality until the release of elixir in 2 years.
You can, ofc, just call the erlang lib yourself.
So on this one, you get the JIT or my float_to_string pretty printer speedup for free. But the Map module cannot use the new maps functions for 2 years.
VM Performance improvements - normally this is minor but for this release it's a big big thing
Bug fixes - to standard library and VM
New Erlang APIs to use directly - you can call Erlang functions easily from Elixir, allowing you to leverage new standard library features, but they won't be used by Elixir because of the compatibility guarantees discussed elsewhere here
Synthetic benchmarks of the Elixir GraphQL implementation show a 64% improvement! Very excited to see how this plays out with real load, but this is very promising.
I just went to leave a slack message to my team about your report in the middle of the night, and discovered that another teammate had already just excitedly mentioned it. Thanks for what you do!
Compiler warnings and errors now include column numbers in addition to line numbers.
I remember reading this warning in the the Absinthe book (pg 90).
"Notice that the column value in the error is 0. Due to a current limitation of the lexer that Absinthe uses (flex)."
Do you think this will be solved as well?
OTP23 — 467.36s
OTP24rc1 — 296.16s
So nice!
The EEP dives right into the problem of "late replies" with little detail explaining the concept. I haven't ran into the problem the EEP is trying to solve but I am interested in understanding it. Can anyone provide more details on how a late reply emerges during gen_server client/server message passing and how an alias is intended to solve the issue? Is this only a problem with distributed messages?
It looks like "late replies" are a problem under an additional constraint that's mentioned in the Motivation section,
"When the author of the client code has full control over the client process such late replies can be handled without a proxy since the code can be aware of these potential stray messages and drop them when received."
So.. you have a server sending messages over distributed erlang to a client that the author doesn't control which can risk receiving late messages if a connection is lost? Ugh.. what?
I tried thinking about this from the proxy process angle as a solution to this mysterious problem,
"Since the proxy process is not alive, a reply will be silently dropped and no stray message will reach the previous client of the request."
But, if a "connection lost" event occurs, why would a message make it to the client to begin with? Why is the client still alive? If the connection is re-establish, then why drop the message?
Wish the EEP more clearly articulated the class of bugs / issues its intending to address and when they occur.
If the call was part of a library function, you can return {error, timeout} or whatever, but the process you ran in will have that reply message in its mailbox, possibly forever if it only does selective receives. This will show up in your system as growing memory use, and slowing message processing (unless all of your messaging is selective receives with new references that benefits from the mailbox marking optimization).
In the proxy process alternative, you would spawn a new process to do the gen_X:call, and message back to the real client process. Any late replies get discarded as the spawned process will be dead. But spawning a process isn't free, even if an Erlang process is much lower cost than an OS thread or process. Without looking at details of the implementation, using a process alias seems like a reasonable optimization.