likely during migration author will use more of iac like terraform, nix, kustomize. so it will reduce their deps on specific cloud. so they can switch faster to any cloud and use tf modules which wraps same things from clouds.
It's not about the specific resources for the most part, it's about knowing what needs replacing and why things are where they are, all while keeping your migration under version control.
Also to be fair, with good enough module abstractions and generic enough problems you can completely replace their implementation with few changes to their call sites for things like:
- A module that fully sets up a static site with DNS, CDN on an S3-compatible service
- A module for setting up an OCI image repo, and spinning resources that runs an image from it as a daemon somewhere.
- A module for setting up an OCI image repo, and sets up a "cron" job to run a binary off of it.
Sure the migration is unlikely to be as clean as terra[grunt|form] down && terra[grunt|form] up but it can get close.
This sounds good in theory. No terraform code I've seen has had that level of abstraction. It's basically copy-pasta everywhere! Maybe I've just had bad luck taking over poorly done, pre-existing projects. Could you point me to a well written example?
There is a lot to be said for just having 100% Terraform defs for everything, sure you rewrite the file from scratch to move to a different cloud, but you know what needs to be in the file
I've written code like that for clients (and specifically executed a transition from bare metal+shell scripts to EKS for a "generic backend" module once), but I can't publish any of it for unfortunate obvious reasons. But FWIW I think it's inherent to IaC to involve a lot of copy-pasta, sometimes your best abstraction will just lead to shifting the copy-pasta from code to call-sites, which tends to be more maintainable but still.
What I can recommend is to treat your modules for what they are: Public library functions. Meaning common sense advice for those applies:
- Approach writing modules by actually designing an API for it. Don't treat it as a folder to dump resources in.
- Make your API as small as you can get away with.
- Its variables and outputs should be, whenever possible, describe a generic black box that could have its internals swapped.
- Make modules composable where you can. Being able to pass around entire module instances in variables because they have clean, stable APIs is very nice.
- Accept some things will inevitably leak but put in the effort to minimize them.
This blog post and the book from the authors (Terraform Up & Running) has some good ideas that got me started with the basics, but there's not much written with regards to more involved usecases: https://blog.gruntwork.io/how-to-create-reusable-infrastruct...
I like your advice but this misses the MOAT of the cloud which is being able to utilize the IaaS/Papas they provide. For example if you are building in azure today going with SQL doesn't make sense because Cosmos and service bus/event grids handle that. This locks you into the vendor bad makes it near impossible to leave.
That's the real cost of cloud: lock in. It's cheap to get in, expensive to get out. Example: You get into data into DynamoDB, it's going to be very difficult to migrate out. Consider all your code is likely using AWS-specific APIs, unless your devs were smart enough to build some sort of abstraction. (I've only seen that done right once.)
I don't think it makes sense to use the abstraction. The cloud is not worth it for that IMO. If you are going to do that using a traditional data center model makes more sense otherwise you can't realize the full value. Also all the providers will have different services or lack of. Even worst, you use a mix of SaaS to avoid the reliance. Better to pay the operations cost.
That's a great point. I look at it from an ops perspective and not the headache the devs deal with. I see where you are coming from. Thanks for the insight.
As we tried to start the migration, we found out that's it not so trivial. I initially hoped that the Kubernetes part could be easily migrated. Not really. We learned that things like Load Balancers should be configured differently, but that's fine.
The biggest problem is that we have to reconsider the build process because the Docker registry, CI, and everything related are on GCP too. Then comes the fact that we built internal db on Bigtable, which we couldn't just switch to DynamoDB. So I guess it will be a slow process of restructuring everything to be free from vendor lock-in.
Decentralization is the only way it's safe out here. Once you give this much power to a monopoly we just look like insects to them. I've moved most of my accounts outside of Google that if I were to lose them it'd be a shame but not a total loss for me. I could then rebuild. Support the little guy today and you will have immunity by decentralization.
Honestly, AWS and Microsoft both seem pretty safe right now. This is pretty unique to Google. I keep warning people never to use Google for B2B, and I keep seeing people get burned.
Decentralization means there's a fire ever month, as some startup goes under or pivots. AWS, Azure, and Office 365 seem to take business continuity seriously.
> Decentralization means there's a fire ever month, as some startup goes under or pivots.
Not sure that follows. Startups are often chasing the kind of hockey stick growth only seen where monopolizing markets (or at least their profits) is possible.
Upvoted. Love that you are not suggesting that another (worse?) monopoly (the government) has to come and save us poor devs. We have options and in reality Google is not a monopoly.
Without government enforcing a competitive market the inevitable consolidation will just continue to create and entrench monolpolists. Incentives tend towards one (or a few) winner takes all.
that is very wrong though. Governments or other monopolies don't enforce free markets. People are free by default and act as such, unless someone forces them otherwise (a big government).
25 comments
[ 2.4 ms ] story [ 57.2 ms ] threadIn any code I've seen in the wild, all resources are coded to a particular cloud provider.
Also to be fair, with good enough module abstractions and generic enough problems you can completely replace their implementation with few changes to their call sites for things like:
- A module that fully sets up a static site with DNS, CDN on an S3-compatible service
- A module for setting up an OCI image repo, and spinning resources that runs an image from it as a daemon somewhere.
- A module for setting up an OCI image repo, and sets up a "cron" job to run a binary off of it.
Sure the migration is unlikely to be as clean as terra[grunt|form] down && terra[grunt|form] up but it can get close.
What I can recommend is to treat your modules for what they are: Public library functions. Meaning common sense advice for those applies:
- Approach writing modules by actually designing an API for it. Don't treat it as a folder to dump resources in.
- Make your API as small as you can get away with.
- Its variables and outputs should be, whenever possible, describe a generic black box that could have its internals swapped.
- Make modules composable where you can. Being able to pass around entire module instances in variables because they have clean, stable APIs is very nice.
- Accept some things will inevitably leak but put in the effort to minimize them.
This blog post and the book from the authors (Terraform Up & Running) has some good ideas that got me started with the basics, but there's not much written with regards to more involved usecases: https://blog.gruntwork.io/how-to-create-reusable-infrastruct...
As we tried to start the migration, we found out that's it not so trivial. I initially hoped that the Kubernetes part could be easily migrated. Not really. We learned that things like Load Balancers should be configured differently, but that's fine.
The biggest problem is that we have to reconsider the build process because the Docker registry, CI, and everything related are on GCP too. Then comes the fact that we built internal db on Bigtable, which we couldn't just switch to DynamoDB. So I guess it will be a slow process of restructuring everything to be free from vendor lock-in.
Decentralization means there's a fire ever month, as some startup goes under or pivots. AWS, Azure, and Office 365 seem to take business continuity seriously.
Not sure that follows. Startups are often chasing the kind of hockey stick growth only seen where monopolizing markets (or at least their profits) is possible.
14 years for Igor to realise that he was The Product. Loyalty counts for nothing if you're not paying, and even then, it's not guaranteed.