How do you manage your CloudFormation processes?

7 points by dmak ↗ HN
I'm curious to know the best practices or how everyone manages their process and CI pipelines for CloudFormation.

Some common problems I struggle with in AWS CloudFormation:

- Only 1 set of changes for a stack can be run at once. This means if multiple devs are deploying changes then they have to wait and incorporate previous changes before they can provision their templates.

- Not more than 1 person can be testing on staging at once. If Dev A successfully provisions resources and Dev B didn't pull new changes then Dev B may undo Dev A's work. This commonly happens in staging.

- There's so much iterative work required to test if a template provisions everything successfully. What may work in a sandbox may not work in staging. Testing in staging seems necessary, but leads to the previous problems.

- Some rollbacks around ECS may hang for hours before timing out. Thankfully, this has only happened in staging, but I don't believe there was a kill switch? I think there was a way to stop the update from the AWS console, but granting access to that defeats the purposes of having a CI to manage your provisioning.

Thoughts and suggestions?

4 comments

[ 5.2 ms ] story [ 17.6 ms ] thread
Dump raw CloudFormation and switch to the CDK. The developer experience is vastly better.
It would be time consuming to migrate everything. How did your team switch over?
I've had good experiences with Sceptre (https://sceptre.cloudreach.com/2.6.3/) for small to medium projects. Puts a bit more structure around the whole process and gives you a few helpful abstractions for deploying changes.
Thanks! I'll check it out