Ask HN: How do you iterate quickly on table schemas?
When working on a new app I don't always know the exact DB schemas I need. Maybe my mind is mush, but I have a feeling lots of devs deal with this. What's your preferred way of working in this scenario? Use NoSQL and just throw everything into objects? Just write a bunch of `ALTER TABLE` commands? Let an ORM handle it?
18 comments
[ 7.4 ms ] story [ 361 ms ] threadUpdateCustomer_1
UpdateCustomer_2
Versioning and compatibility are of course concerns but that is true of any scheme you come up with.
If you think you haven’t exported “business logic” to your data store in any case you are wrong.
My build process creates an artifact and that artifact gets deployed through the entire pipeline - Dev/Qa/UAT/Prod. There is one source of truth.
But when half your code is in stored procedures and the other half is in code, how do you keep them in sync? If I need to redeploy an older version, it's a simple matter of redeploying the artifact from v1. How do you make sure the code stays in sync with the stored procedure?
I can honestly say, that any system I've designed from scratch has no business logic in the data store.
Those might be business rules you are ok with having in your data store, but they are a point of sync between your code and the store that you have to manage.
If your stored procedures are their to abstract your table structure for write operations they act as a way to make that sync easier not harder.
As far as writing your domain model to a relational schema, you should be doing that as a single module (in process) or micro service (out of process) that all other code calls. Meaning changing the representation of a domain model should only require a change in one place. For a monolith it means redeploying one artifact. For a microservice, it means deploying that microservice.
Even better with a nosql solution like Mongo, if you are using a strongly typed language. Your domain model is your schema. You change your schema and the data model changes.
With some ORMs you can just change your object and it creates alter table statements for you - I personally don't trust them but it is an alternative.
After that, I don't change my database much. But when I do, I prefer vi and psql:
Whenever I want to make a change, I use vi to edit tables.sql. Then I run psql and its \i command to run the stuff in the file --- which drops the schema and all its tables, and recreates everything again. I don't know if any other database treats schemas as Postgres does though, as folders for tables.By the way, you only want to do this on your test database, while making it, before users start using it, because it drops all the tables and their data. After you've gone live, it's alter table, baby.
Even after go-live this technique is useful for idempotent commands like creating views and functions. But instead of the drop-schema-cascade command, you have to put commands at the top to drop each of the views or functions, usually in the opposite order you create them, in case there are any dependencies.
If you like GUIs or are new to SQL, this way is daunting. I think it is the happiest way long term, though.
To answer the question as asked, I use pgadmin or the corresponding database management tool to update the database. My first actual migration is a stable version of the database.
First week or so: Change the DB as much as you like, directly, using the visual tools or create/alter statements as you see fit.
Once you're ready to do things right: Use change scripts. Save every create/alter statement, as well as data population and modification queries into a .sql file in a directory, named sensibly so that you can run them in order to exactly reproduce your schema at any point, on any machine.
Important to note: This is not hard. SQL is not hard. Avoid tooling written by people who think SQL is hard, that therefore build elaborate workflows that are in fact hard in an effort to shield you from having to touch the database.
That being said, when you don't know what your schema is going to look like, why not use a Nosql solution - either a pure nosql solution or the JSON support built into modern databases? If you write your code correctly and have one module retrieve for one domaim, you should be able to isolate changes.
But, if I do have to use Sql, I agree, put all of your changes in sql files numbered and have a "database version" and find/create tooling that can automate running the Sql scripts and run it as part of your CI/CD.
Because one day soon, you will know what your schema is going to look like. And the next 10 years of your business will be a lot more pleasant had you optimized for them rather than the first week of greenfield development.