That’s pretty sweet. Makes me think about software upgrade scenarios. A typical upgrade process for long-running services I’ve seen is usually some variant of “flush everything out to disk; shutdown process; run some hand-crafted upgrade code for the on-disk datastructures; start the new process.”
On the other hand, the development for this stuff usually consists of a sequence of small patches. You could generate a lot of that on-disk upgrade code by applying something like this against all those patches. Maybe you could take it a step further and never actually tear down the process (by doing that memory mapping stuff the article touches on).
I wonder how far you can take that concept. People develop software today with an edit -> build -> deploy cycle. With sufficient tools, could one do development solely by describing how to patch the running process? Your repository consists of a sequence of such patches: CI applies the patches against a running process (and then runs all your tests). If that passes, you can deploy the patch everywhere using an identical procedure.
Fixes the issue of customers running outdated software because they can’t handle the one minute of downtime associated with an upgrade to a critical service. You don’t need to keep years-old upgrade code in the codebase (and keeping it well-tested) because one of those customers might want to upgrade someday. On the other hand, pulling bad patches caught late in the cycle becomes more difficult. And also the problem of downtime already has workarounds today, usually by making use of redundancy (which remains valuable even outside of upgrades).
I dunno, it’s fun to think about where and how you could apply this. Hot-reloading structs is a totally novel idea for me :-)
Unless I'm missing something, what you propose doesn't solve the problem at all. The file on disk is the canonical representation of the application source code. The canonical representation is what has to be upgraded since processes don't live forever (processes are effectively an ephemeral copy of the canonical code). That's the primary reason for the shutdown/upgrade/restart flow.
The parent meant “what if the file on disk contains code deltas that you need to apply to the executable”.
It’s not just deltas, it’s structural modifications of the code that allows it to be compatible with the old version.
I think we partially have this with tv the likes of task runners (eg Celery in python) where you can reload the workers (code instances) whole having the old workers running.
This was always hailed as one of the great features of Lisp-based languages, especially Common Lisp. Everything old is new again? Or is CL just too clunky and Scheme just too weird?
8 comments
[ 1.8 ms ] story [ 31.4 ms ] threadOn the other hand, the development for this stuff usually consists of a sequence of small patches. You could generate a lot of that on-disk upgrade code by applying something like this against all those patches. Maybe you could take it a step further and never actually tear down the process (by doing that memory mapping stuff the article touches on).
I wonder how far you can take that concept. People develop software today with an edit -> build -> deploy cycle. With sufficient tools, could one do development solely by describing how to patch the running process? Your repository consists of a sequence of such patches: CI applies the patches against a running process (and then runs all your tests). If that passes, you can deploy the patch everywhere using an identical procedure.
Fixes the issue of customers running outdated software because they can’t handle the one minute of downtime associated with an upgrade to a critical service. You don’t need to keep years-old upgrade code in the codebase (and keeping it well-tested) because one of those customers might want to upgrade someday. On the other hand, pulling bad patches caught late in the cycle becomes more difficult. And also the problem of downtime already has workarounds today, usually by making use of redundancy (which remains valuable even outside of upgrades).
I dunno, it’s fun to think about where and how you could apply this. Hot-reloading structs is a totally novel idea for me :-)
It’s not just deltas, it’s structural modifications of the code that allows it to be compatible with the old version.
I think we partially have this with tv the likes of task runners (eg Celery in python) where you can reload the workers (code instances) whole having the old workers running.
Not sure if this got any traction any further.