If only you could take and restore snapshots easily in this platforms. I use used nested virtualization with qemu or vbox to avoid having service accounts, automation scripts, functions,etc...
But this is a great step forward, I have seen (including with myself) heavy usage of cloud VMs as workstations.
I do wonder how big the use case for suspend/hibernate is in practice. Are there that many services that (a) see infrequent use that justifies this cost reduction and (b) can't just be booted fresh when needed. I could see if there's a lot of compute-intensive data cached, but this could also just be stored in a disk-persistent cache.
It’s a useful feature in some cases for certain stateful workloads (e.g. cloud-based workstations) but it’s almost always 100% better architecturally to develop systems that can just be thrown away and rebuilt vs needing to preserve state.
True. We do a fair bit of cloud-based dev environments and I’d be worried about risks from tasks getting paused/resumed that aren’t well-designed for it. In theory this could cause undesirable outcomes but it’s probably not an issue in practice.
I've used it extensively for hosting the heavily-modded Minecraft servers we play with. Those things can take easily 5-10 minutes to boot up fresh, as opposed to the very quick resume.
We have been using GCP hibernation since 2018 too, but it was alpha back then. The AWS version also had some limitations (it only supposed Amazon Linux at the time) - current prerequisites here: https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/hibernat...
We have hundreds of developer K8s clusters that get used during a normal workday and get shutdown later. Because there are many services (each with their own readiness criteria), a cold boot takes a long time until all services stabilize (worst case I've seen was over 30 minutes). With hibernation they can resume where they started off within a couple of minutes. During development it's often the case that one may be doing experiments and changing the current state, so preserving everything is a plus.
If, as it was often the case when it was alpha, hibernation failed, then the normal startup procedure would take place.
We don't use it in production but I don't think it's inconceivable that there are other use-cases where preserving state could shave time in getting resources ready.
> Because there are many services (each with their own readiness criteria), a cold boot takes a long time until all services stabilize (worst case I've seen was over 30 minutes). With hibernation they can resume where they started off within a couple of minutes.
This is exactly what we use hibernation for in conjunction with EC2 Warm Pools -- fast autoscaling of services that have long boot times. There's an argument to be made that fixing slow boots should be the "correct" solution, but in large enough organizations, hibernated instances are a convenient workaround to buy you some time to navigate the organizational dynamics (and technical debt) that lead to the slow boot times in the first place.
At work we have this for stablediffusion and similar servers, I think one runs at .5€/h or more. Automatic start and stop outside of office working hours.
It even makes sense if you have on-prem bare metal, with electricity prices being as insane as they are, you'll notice the effect pretty much immediately.
I have a ci/cd machine that is only useful to have on when I'm actively working on the projects it manages, makes the difference between a few bucks a month and ~$80
It's been a decade since I used VMWare and I'm still waiting for Google, Amazon, etc. to catch up (Live Migration, Fault Tolerance, Instant Clone). Unfortunately all we've gotten is 15 different ways to run a docker container.
This is old. Suspension isn't transparent to the VM which reduces its usefulness.
> When you make a suspend request, you send an ACPI suspend signal to the VM instance. If the VM does not respond to the ACPI S3 suspend signal within a couple of minutes, Compute Engine cancels the suspend attempt and returns the VM to a RUNNING state.
> When you initiate hibernation, the instance moves to the stopping state. Amazon EC2 signals the operating system to perform hibernation (suspend-to-disk). The hibernation freezes all of the processes, saves the contents of the RAM to the EBS root volume, and then performs a regular shutdown.
I guess that time keeping could go crazy if you don't signal the OS to suspend or something like that. I remember some instances (years ago) where a VMWare live migration would make a VM clock jump or do something weird, and the guest OS would... "get scared".
I wish more programming languages had this - would make it much easier to program simulations because agents could be written in the same language, instead of some other preemptible DSL, it'd also enable reproducible debugging and "system-jumping" processes if the suspended state could be serialized
23 comments
[ 4.4 ms ] story [ 57.6 ms ] threadSurprising given that NVidia has vGPUs capable of hibernation:
https://docs.nvidia.com/grid/15.0/grid-vgpu-release-notes-mi...
"vGPU hibernation is supported on all supported GPUs, Microsoft Azure Stack HCI hypervisor software releases, and guest operating systems."
But this is a great step forward, I have seen (including with myself) heavy usage of cloud VMs as workstations.
I do wonder how big the use case for suspend/hibernate is in practice. Are there that many services that (a) see infrequent use that justifies this cost reduction and (b) can't just be booted fresh when needed. I could see if there's a lot of compute-intensive data cached, but this could also just be stored in a disk-persistent cache.
We have been using GCP hibernation since 2018 too, but it was alpha back then. The AWS version also had some limitations (it only supposed Amazon Linux at the time) - current prerequisites here: https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/hibernat...
We have hundreds of developer K8s clusters that get used during a normal workday and get shutdown later. Because there are many services (each with their own readiness criteria), a cold boot takes a long time until all services stabilize (worst case I've seen was over 30 minutes). With hibernation they can resume where they started off within a couple of minutes. During development it's often the case that one may be doing experiments and changing the current state, so preserving everything is a plus.
If, as it was often the case when it was alpha, hibernation failed, then the normal startup procedure would take place.
We don't use it in production but I don't think it's inconceivable that there are other use-cases where preserving state could shave time in getting resources ready.
This is exactly what we use hibernation for in conjunction with EC2 Warm Pools -- fast autoscaling of services that have long boot times. There's an argument to be made that fixing slow boots should be the "correct" solution, but in large enough organizations, hibernated instances are a convenient workaround to buy you some time to navigate the organizational dynamics (and technical debt) that lead to the slow boot times in the first place.
It even makes sense if you have on-prem bare metal, with electricity prices being as insane as they are, you'll notice the effect pretty much immediately.
I have a ci/cd machine that is only useful to have on when I'm actively working on the projects it manages, makes the difference between a few bucks a month and ~$80
> When you make a suspend request, you send an ACPI suspend signal to the VM instance. If the VM does not respond to the ACPI S3 suspend signal within a couple of minutes, Compute Engine cancels the suspend attempt and returns the VM to a RUNNING state.
> When you initiate hibernation, the instance moves to the stopping state. Amazon EC2 signals the operating system to perform hibernation (suspend-to-disk). The hibernation freezes all of the processes, saves the contents of the RAM to the EBS root volume, and then performs a regular shutdown.