Dolt (https://github.com/liquidata-inc/dolt) and DoltHub (https://dolthub.com) are Git and GitHub for data. Having a versioned database makes collaborating on data more fluid and reliable in the same way that Git improves source code collaboration for software engineers.
This blog discusses DoltHub’s newest feature, cell history inspection. You can inspect the changes between versions of every cell in every table with just a click, making debugging between versions easy to trace.
My problem with this is the same as every other such systems: changes are tracked at the cell or row level, and not transaction level. So if I issue a SQL query fixing a column, for example changing 'na' values to NULL, and someone simultaneously makes changes that make a cell 'na', after a merge those cells will still be 'na'.
So in reality, every time I merge something I need to go over and manually re-apply my bulk queries if needed. While the UX is much better than putting CSV files in Git, Dolt does not have a better understanding of data than Git, and does not help with common data transformations either.
That is not true. Dolt will throw a merge conflict if two cells are modified on two different branches. You still have to pick which one but at least you know.
So will Git. My problem is if that specific cell was not 'na' when the batch change was applied. In that case, there is no conflict.
If those were SQL transaction, one would abort and re-try, giving you the correct result. Even when using a transaction model as simple as Redis, you would get a conflict if you issued the correct `WATCH` statements.
4 comments
[ 3.3 ms ] story [ 19.2 ms ] threadThis blog discusses DoltHub’s newest feature, cell history inspection. You can inspect the changes between versions of every cell in every table with just a click, making debugging between versions easy to trace.
So in reality, every time I merge something I need to go over and manually re-apply my bulk queries if needed. While the UX is much better than putting CSV files in Git, Dolt does not have a better understanding of data than Git, and does not help with common data transformations either.
If those were SQL transaction, one would abort and re-try, giving you the correct result. Even when using a transaction model as simple as Redis, you would get a conflict if you issued the correct `WATCH` statements.