Do you have any benchmarks with a mix of long open transactions and short ones? I've struggled a lot with WAL-E in the past there, and am curious if that changes here.
I must say I'm quite pleased to see how well Go version works. It does only use 1.5x the CPU and (predictably) much more RAM/VRAM, but not a crazy amount either (the expected increase is 2x).
Of course you can write a more optimal version in C / C++ / Zig / Rust, but at the same time Go is much easier to write and you don't pay for the convenience with an absurd performance loss like in Python or PHP
In many cases it could even be more optimised, but somehow using value types is a lost art.
Granted Go could make it much more easier to understand when they escape the stack into the heap, as many of their design decisions, simple and easy aren't the same.
Quick one on the benchmark: was the 2.8GB peak virtual or resident? Go reserves a large virtual arena it mostly never faults in, so RSS tends to be a fraction of the virtual peak, and if Postgres headroom was getting squeezed off the virtual number you were sizing against memory the kernel never actually charges for.
As someone who only has a cursory knowledge of Postgres backup systems, how does this compare to something like pgBackRest? When would someone reach for one over the other?
If you're running pg yourself I recommend pgbackrest. It doesn't run as a daemon, & it forks multiple processes for concurrency. But it's simple to run as archive_command & is light on resources outside concurrency
The article’s distinction about memory is a bit too simplified. Virtual memory is address space mapped or reserved by the process. RSS is resident physical pages, not necessarily “actively used.”. RSS also overcounts shared memory, which is why PSS/USS matter.
11 comments
[ 5.7 ms ] story [ 40.8 ms ] threadOf course you can write a more optimal version in C / C++ / Zig / Rust, but at the same time Go is much easier to write and you don't pay for the convenience with an absurd performance loss like in Python or PHP
Granted Go could make it much more easier to understand when they escape the stack into the heap, as many of their design decisions, simple and easy aren't the same.
wal-g/wal-rus have higher throughput