Git filter-repo is mainly from a single author newren, who also is a significant contributor to git itself: https://github.com/git/git/graphs/contributors
The repo seems open to PR's though, given the many commits from one-time contributors.
[Edited]
A question: how would one check the integrity of the resulting git repo (apart from an obvious rsync check of the source files in master)? Say, I would like to create a branch from an older commit and rewrite its history with "git rebase -i"; that could fail on a a repo without full integrity.
This looks so much nicer than filter-branch, which is slow beyond belief.
We've used bfg (which at least performs in reasonable time) to filter out files containing viruses or security tokens/passwords, but there are some definite limitations.
> If commit messages refer to other commits by ID (e.g. "this reverts commit 01234567890abcdef", "In commit 0013deadbeef9a..."), those commit messages should be rewritten to refer to the new commit IDs
Finally, a tool that does this! I wish git rebase could do it.
I used this earlier this year to split a monorepo into a submodule + 4 other repos. It made it incredibly easy to rewrite history to move files going to different repos, then remove the history of any files not staying in a particular repo.
And it was fast. Also, being able to pass in files containing rewrite rules allowed me to easily do multiple dry-runs of the code split ahead of time so I was able to minimize the code freeze for our team.
I did something similar a couple years ago. I needed to rewrite an entire repo's history so that I could reformat all the code _and_ modernize all the JS syntax throughout every commit in the repo's history (dating back to 2013).
I started with `filter-branch`, but found it way too slow (especially on a corporate-controlled Windows machine, where starting up additional processes seems to have a lot of overhead). I concluded that I needed to run the entire filtering logic in a single process to avoid that overhead. Started writing my own with `pygit2`, but then found a repo called `pylter-branch` which did most of that same "loop through commits and reprocess" work for me - I just had to add a lot of additional logic on top for the specific reprocessing I wanted to do.
Ended up being able to reprocess about 15K commits in around 4.5 hours. Given the amount of processing I was doing, that was pretty good.
I did an extensive writeup [1] on the problem statement, investigation, and techniques I used if anyone's interested.
Heh. It wasn't even the rest of my team - I'm the only one who cared enough to do anything about it :)
Per the post, I wanted to auto-reformat the entire codebase, as well as codemod the JS source to update it to more modern syntax (ES modules instead of AMD, `const/let` instead of `var`, shorthand object literals, etc). But, that would have left most of the codebase history showing that the last commit touching a line was "MARK REFORMATTED ALL THE THINGS".
Since I was rewriting the entire repo history to strip out all the junk files that were consuming space, which was going to create an entirely new alternate history anyway, I figured I might as well take advantage of the opportunity to redo all the source as well:
9 comments
[ 3.4 ms ] story [ 45.2 ms ] thread[Edited] A question: how would one check the integrity of the resulting git repo (apart from an obvious rsync check of the source files in master)? Say, I would like to create a branch from an older commit and rewrite its history with "git rebase -i"; that could fail on a a repo without full integrity.
We've used bfg (which at least performs in reasonable time) to filter out files containing viruses or security tokens/passwords, but there are some definite limitations.
Finally, a tool that does this! I wish git rebase could do it.
And it was fast. Also, being able to pass in files containing rewrite rules allowed me to easily do multiple dry-runs of the code split ahead of time so I was able to minimize the code freeze for our team.
I started with `filter-branch`, but found it way too slow (especially on a corporate-controlled Windows machine, where starting up additional processes seems to have a lot of overhead). I concluded that I needed to run the entire filtering logic in a single process to avoid that overhead. Started writing my own with `pygit2`, but then found a repo called `pylter-branch` which did most of that same "loop through commits and reprocess" work for me - I just had to add a lot of additional logic on top for the specific reprocessing I wanted to do.
Ended up being able to reprocess about 15K commits in around 4.5 hours. Given the amount of processing I was doing, that was pretty good.
I did an extensive writeup [1] on the problem statement, investigation, and techniques I used if anyone's interested.
[0] https://github.com/sergelevin/pylter-branch
[1] https://blog.isquaredsoftware.com/2018/11/git-js-history-rew...
Per the post, I wanted to auto-reformat the entire codebase, as well as codemod the JS source to update it to more modern syntax (ES modules instead of AMD, `const/let` instead of `var`, shorthand object literals, etc). But, that would have left most of the codebase history showing that the last commit touching a line was "MARK REFORMATTED ALL THE THINGS".
Since I was rewriting the entire repo history to strip out all the junk files that were consuming space, which was going to create an entirely new alternate history anyway, I figured I might as well take advantage of the opportunity to redo all the source as well:
https://blog.isquaredsoftware.com/2018/11/git-js-history-rew...
Indeed, this is a good reason to do as you chose. I hadn't thought of it (even though I have done the same to a code base).