This hit me when moving from a start-up to a FAANG. There is effectively an infinite amount of work for me to do on any given day, so at some point I just have to decide to stop - if I don't, I'll just end up tired…
Solving storage at the tens of TB scale with commodity hardware is fine to a point (I have a ZFS NAS at home) but has much more ongoing maintenance burden than S3 and you need at least 2 copies for it to be a remotely…
> Are you saying we should stick with AWS because most stick with AWS? Broadly speaking yes - there is a lot of value in having a deeper pool of skilled people to hire from, and there are enough differences between…
I have heard too many horror stories about Ceph (and OpenStack) to be confident about that. I certainly don't think I can truly beat S3 on cost or performance at the terabyte scale for household data - and while larger…
Those are fair points - I have seen truly spiky workloads like that very occasionally, but more often those spikes are a precursor to more sustained usage in a similar manner and so would quickly warrant hardware…
As an addendum to this: if you absolutely must use cloud, stick with AWS. Using Azure is (IMO) a fucking miserable experience and their only advantage (InfiniBand) is better served by buying your own hardware. GCP and…
I work in HPC for a cloud provider, and fully endorse this move. Anonymously, of course. You can make an economic argument for or against cloud in practically every IT domain, but in HPC the case for on-prem is really…
Anything by Rooster Teeth is worth a try for comedy; not to everyone's taste, but if you like the main RT podcast there is a long backlog worth listening to. Skipping back 200 episodes got me through my PhD write-up.…
This hit me when moving from a start-up to a FAANG. There is effectively an infinite amount of work for me to do on any given day, so at some point I just have to decide to stop - if I don't, I'll just end up tired…
Solving storage at the tens of TB scale with commodity hardware is fine to a point (I have a ZFS NAS at home) but has much more ongoing maintenance burden than S3 and you need at least 2 copies for it to be a remotely…
> Are you saying we should stick with AWS because most stick with AWS? Broadly speaking yes - there is a lot of value in having a deeper pool of skilled people to hire from, and there are enough differences between…
I have heard too many horror stories about Ceph (and OpenStack) to be confident about that. I certainly don't think I can truly beat S3 on cost or performance at the terabyte scale for household data - and while larger…
Those are fair points - I have seen truly spiky workloads like that very occasionally, but more often those spikes are a precursor to more sustained usage in a similar manner and so would quickly warrant hardware…
As an addendum to this: if you absolutely must use cloud, stick with AWS. Using Azure is (IMO) a fucking miserable experience and their only advantage (InfiniBand) is better served by buying your own hardware. GCP and…
I work in HPC for a cloud provider, and fully endorse this move. Anonymously, of course. You can make an economic argument for or against cloud in practically every IT domain, but in HPC the case for on-prem is really…
Anything by Rooster Teeth is worth a try for comedy; not to everyone's taste, but if you like the main RT podcast there is a long backlog worth listening to. Skipping back 200 episodes got me through my PhD write-up.…