Using the expand and contract pattern for schema changes (prisma.io)

I use this example when I speak about and teach devops trainings.

I call it the migration sandwich. (Nothing to do with the cube rule).

A piece of bread isn't a sandwich and a single migration in a tool like alembic isn't a "sandwich" either. You have a couple layers of bread with one or several layers of toppings and it's not a sandwich until it's all done.

People get a laugh out of the "idiot sandwich meme" and we always have a good conversation about what gnarly migrations people have seen or done (72+ hours of runtime, splitting to dozens or more tables and then reconstructing things, splitting things out to safely be worked on in the expanded state for weeks, etc).

I had never heard it called "expand and contract" before reading this article a few years ago.

What does everyone else call these?

I’m confused. I thought Expand and Contract was about mutating an existing schema, adding columns and tables, not creating a full replacement schema. But maybe I misunderstood?

What’s in the article, I know as the Strangler Fig Pattern.

This is just the natural solution that falls out if you want to change a schema with no downtime. I always just called it “dual writing”.

[dead]

This is the model we used at the SaaS I worked for a decade ago. It worked great to allow for smooth, zero-downtime upgrades across a fleet of thousands of DB servers serving tens of thousands of app servers and millions of active users.

Expand Contract from Fowler's bliki

https://martinfowler.com/bliki/ParallelChange.html

Is there any easy way to implement this pattern in AWS RDS deployments where we need to deploy multiple times a day and need it to be done in few minutes?

Ok hear me out. What if this whole process was statefully managed for you as an add on to your database?

Like you essentially defined the steps in a temporal like workflow and then it does all the work of expanding, verifying and contracting.

I use Prisma on almost all my node.js projects these days, and I wish that part of schema migrations was also automated by Prisma. But last I checked, it doesn't even rename columns properly.

I feel like maybe they should invest more R&D in their migrations technology? The ORM is pretty great.

1) double write essentially

2) migration involves the problem of mixing a migration write with an actual live in flight mutation. Cassandra would solve this with additional per cell write time tracking or a migrated vs new mutation flag

3) and then you have deletes. So you'll need a tombstone mechanism, because if a live delete of a cell value is overwritten by a migrated value, than data that is deleted comes back to life

im not sure about the name, but this is a great little doc for introducing junior devs to migrations.

the only thing i would add is a minor and major version changes, so its clear how the different class ent stages are labeled/how you track when you're ready to backfill

The following academic work (132 page pdf) elucidates this pattern in the context of a real application:

https://ris.utwente.nl/ws/portalfiles/portal/275963001/PDEng...