Fossil's new(ish) semi-automatic bidirectional interaction with Git mirrors has finally made Mercurial replaceable for me. However, it is hard to ignore that the IT world has mostly decided to settle with whatever is the most commonly used right now, no matter if it is actually the best solution - so at least when it comes to DVCS, the war seems to be over.
Exporting to Git: You set a "Git mirror" once, optionally with a remote URI, and call fossil's export routine. Example: I wrote a cronjob that runs once every night that does nothing but update my Git mirrors from the current Fossils.
cd $FOSSILDIR
for f in *; do
/usr/local/bin/fossil git export -q -R $f
done
But this doesn't hold for things like languages, adoption of new tech over old (no matter how prominent it is) happens all the time, but it usually starts at the small scale, like start-ups. It definitely takes a hell of a long time, and it would probably never usurp 100% of gits usership (or even 50%), but it will probably be used in some capacity. Of course Java's still as big as ever, but we have seen an uptick in relatively new languages still coming into use (I think of things like Elm/Elixir).
There was always an "uptick in relatively new languages coming into use", as long as I remember. The problem is that most of them die before they mature, and some achieve a "peak cool" but then fade into obscurity, or find a relatively small niche. Very few new languages ever get popular enough to get into the top 10 on a permanent basis, and even fewer displace a large player that was previously popular. Meanwhile, most code running on any given computing device is still written in C, C++, or Java.
The article is basically arguing that we're likely to see the same state of affairs in VCS land. Sure, there will still be new ones coming into use, and fading out, or finding small niches - but Git will remain dominant for decades to come.
Does not Fossil’s “no rebases” philosophy bother you?
IMHO, the ability to “commit early, commit often” and to squash/rebase original work later into clean and understandable commits is one of the best features of Git.
Right - and I think that point 6 and 7 are exactly why I use rebase daily, and why I will never use Fossil.
The version control history is made for reading by other people, so it is a story. After all, we only write the commit once, but people will read it many more times. (This is especially true if there are code reviews involved)
I am an imperfect programmer. My work-in-progres is often broken, and even fails to compile. Sometimes I will refactor interface only, and will want to checkpoint my work before I go and refactor implementation as well. Sometimes I will choose a totally wrong approach and revert it later. Sometimes I will disable/break large part of system on purpose, to make testing easier. And I often do stupid data-destroying mistakes, so I want an ability to save/store all the past versions, even if they are completely broken.
My “raw” commits may look like: “start on feature X”, “refactor interface Y”, “more work on feature X”, “wip commit”, “fix tests”, “fix performance”, “fix more tests”. Does any future reader care it took me 3 commits to get the tests right? Do they care that I discovered the need to refactor only while I was halfway in feature X implementation? Do they want to see a repo that won’t even compile? Do they want to hav to cherry-pick dozens of commits to get the tests to pass? I don’t think so.
The final version will only have two commits, “refactor” and “feature X”. It would be obvious to everyone which lines of code are associated with which change. Each revision will be buildable, and will pass all tests - so bisect will actually work.
(If rebase support is missing, it is possible to “fake” it by having multiple checkouts and manually copying files around. But this is much more error prone and dangerous. I have spent plenty of time with SVN/CVS, manually copying files and applying patches - and I can tell that having this integrated with version control is much more pleasant)
Git can easily be replaced in my company if there was business need for it. For now it does do its job, but there is no reason we wouldn't migrate if there was a better alternative. Right now there isn't one.
Perforce is still in use in content heavy industries because Git still struggles with large binaries. Something with git branch semantics that could handle those files would be huge.
I really miss being able to check in chunks instead of full files, and streams are a poor replacement for git branches but it does let you do monorepo. P4V is a buggy mess though.
Another feature of Perforce I liked was view specs. If your dealing with massive repos (as for example games tend to have) you often only want some small portion of the entire repo. You may also want to alter the repo layout on your local machine and a client view spec can allow you to slice and dice the content in many interesting ways.
This is also very useful for working with external contractors. Not only can a user specify their own view specs, there is admin control for client views that can make portions of the repo read-only or effectively invisible on a per-user basis.
I find it ironic that in the same post the author complains about a git monopoly ("Git effectively has a global monopoly on DVCSes") while wishing the same thing for fossil ("Fossil's world domination will not come from...").
"world domination", specifically, refers to a long-running joke about the corporate-sponsored capitalist competitors to FLOSS. See [0] for an example of the joke being used to title a talk and frame business decisions. [1] contains a history of the phrase in the Linux world. Note that Linux and git share the same original author/designer.
SVN seemed irreplaceable not so long ago and CVS before that and RCS before that... When someone comes up with something demonstrably better and gets the right people to start using it, people will switch over.
SVN also had serious difficulties and performance limitations which git does not.
Git is somewhat confusing to use, but not enough so that anyone really cares all that much (besides a few people who really care) and that is not a recipe for easy replacement.
There were/are slightly less confusing version control systems (mercurial) but they didn't catch on for whatever reason.
I mean so is Unix and it's still around all these years. Heck there's nothing especially obvious about a for loop, yet every language has them. Once everyone's accustomed to the weird interface they are just going to demand everything else have the same weird interface they're used to. This is one of the lamentations of the Unix Hater's Handbook.
If you’re talking about a “for (initialize; condition; step) { body }” for loop, most newer languages don’t have them AFAICT.
I would say the other kind of for loop, a “for x in y { body }” loop, is very intuitive and obvious to most English speakers. We use idioms like “for each A, do B” all the time.
Git is probably destined to be Unix for our generation that was born after Unics/Unix invention. It is not a coincident that the designer/programmer of the most popular Unix version/clone of our generation is the same guy who designed and programmed Git.
I would love to see a merger of Pijul and Darcs again. Having two diverging patch-based approaches (one declining, one not lifting off) is not a good thing.
Git doesn't work very well when trying to version control things which aren't text and are large binary files. There's an acceptable workaround which is only a little awkward with git-lfs.
The general consensus is that putting big files in git means you're doing something wrong and the problem is with your environment not git. (or there are special-purpose tools for your kind of workflow which handle the specifics of your use case, like CAD/CAM/etc.)
Workflows like that generally don't fit into nice little boxes anyway the way source code management does.
> The general consensus is that putting big files in git means you're doing something wrong and the problem is with your environment not git.
This is not a “general” consensus. It’s a consensus among hardcore proponents of git. I love git, it has made my life better. I still think it’s large file support story is shitty/suboptimal, and there are valid use cases where a general purpose VCS is used to track large binary assets along code and git would do well to be a general purpose VCS. It’s a limitation of git. It’s not a fatal limitation, and git still has enough benefits (which include availability and mindshare), but it is still an unfortunate limitation and somewhat ironic for a tool born in a world where everything is just a “sequence of bytes”.
I agree with the first half of the statement: we indeed do not have a cloud-first version control system. It could bring some real benefits of tight integration with continuous build and other things considered to be essentials nowadays.
I don't see files going anywhere. I would like to listen to examples of "code-in-database is already halfway here", if you happen to have them.
Low-code is very suddenly going to be a big thing in the next couple years and when it happens there are now 750M more "coders" and those types of people are not interested in git pull --rebase --fucked --whatdidido
I'll believe it when I see it and it's an actual improvement. It's not like "cloud" is some sort of Platonic embodiment of goodness and anything that is "cloudier" than an alternative is automatically better. All the attempts to put code in "databases" I've ever seen make easy things sorta easy and make medium things way harder and you can just forget about hard things; if I can't do the equivalent of "perl -pi -e 's/OldVariable/NewVariable/g' then it just isn't going to displace a file system for me. Not because I need that exact commmand or perl, but for what it represents, the ability to just do things to my code as needed, without needing to wait for someone to implement an API or sluggishly do it over a network. Another example would be just throwing a code analysis tool at my code without needing some sort of permission from my "cloud code provider".
File systems haven't been disappearing, they've just been getting hidden from users. I have seen exceedingly little evidence that anything beyond merely hiding them is actually happening in the industry. "Stick all the files in databases" seems to be seeing no penetration beyond music collections, which at this point, I think we can call a "mere use case" rather than the vanguard of a revolution, since it's a good 10-15 years old minimum.
I am happy to use whatever everyone else starts using, provided it works at least as well. But I also do not have many problems that git won't solve, so I don't feel a burning need to switch. I think git is good enough that source control is no longer a very interesting problem.
When I started developing the options people were taking seriously were Mercurial and git. At first I was using mercurial, but since everyone else was using git I switched to that. So those are the only ones I've seriously used.
I don't think there are many people that considered CVS good enough, even back in the days. For example, there was no way for end-users to retain the history of a file when renaming/copying it. When the FreeBSD project still used CVS, committers always had to file tickets against the admins to ask them to do it server-side through a so-called 'repocopy':
Speaking personally, definitely no. Even before I heard of git, I did not like SVN. In particular, it made an error I don't hear many other people mention where directories were checked out to revisions, rather than entire repositories. Consequently it was very easy to accidentally update just a subset of your repository, and have your whole repository in a state that didn't exist in the version control system at all. There were several times there were builds that were completely unreproducible because it turned out that they were, technically, r83726, except this directory was r84713, but this other directory was r78372 except for a subdirectory that was r84299, etc.
That problem I saw even before I saw a replacement.
But once I saw git, having a full copy of the repo was pretty killer. It was also better at merging, partially for having the history local. In fact I recall some SVN people protesting at the time that merging wasn't so hard, but in a nutshell, they were just wrong. It was harder. Source control that can't merge fluidly is pretty limited from the get-go. There's literally entire dimensions of things we do with git that were so impractical with SVN that only the very largest projects could afford to do them, with the whole "workflow" question. Now you don't even hear about the difficulties of merging since all the open source SCMs copied each other and "good merging" is just table stakes now.
The problem is is that git doesn't really have fatal flaws, it has annoyances, and that's not the same thing. "The UI is difficult" is an annoyance for its target audience. Most every other criticism I've seen of it is an annoyance, not a fatal flaw. Fossil doesn't solve a problem I have with git. It has some neat ideas and arguably does solve some problems, but it doesn't solve the problems I have with git; you could lift Fossil's solutions to wiki and bugtracking and just put them in Git and I'd probably be happier with that than Fossil itself.
The problem isn't that Fossil is an SCM competing with git; the problem is that Fossil is a monolith competing with the rapidly-moving git ecosystem. The former is perhaps beatable, the latter is a juggernaut. All of Fossil's other features sitting in git is something that might get somewhere, but if the base source control isn't git-based, I can't sneak it into a work project to try it out. I've got a corporate mandate that all source belongs somewhere standard, and it's a perfectly sensible requirement for someone paying me to do a job.
I would check, double-check, and triple check whatever use case you miss that for. It is a serious misfeature, and I find any workflow that critically depends on it to be very likely a process smell.
Handling of blobs I can at least see preferring the differences, though git-lfs mostly acceptably shims that for me. The need to use git-lfs is definitely an annoyance, but it's another place where it's probably not a fatal flaw.
Or, to the extent that it arguably is, it would be when you're using git as something that it really, really isn't... it's a source code management tool. It's a good enough source code management tool that it's useful for many other things as well, like the way some people directly drive their personal websites with it. But it isn't a tool for managing lots of large files that can't be textually diffed, not a generic "content" management tool. I wouldn't expect git to necessarily store terabytes of video files, but then, it's not like Fossil is going to do that either from the sounds of it. git-lfs shims content blobs well enough to make it just an annoyance for source code management.
In the case you mentioned of "just a few commits" it's of marginal utility, because you can do that manually almost as quickly, but it's a killer when it comes to "I don't really know when we introduced this" and you have to sift over thousands of commits. Unfortunately, it requires that you have the discipline to keep every commit on the relevant branches buildable (enough so to find the bugs), which is something you can do going forward but can't retroactively apply to a code base very well.
None of the other big VCS had fatal flaws. If they had fatal flaws, they wouldn't have been used.
Sure, not being able to rename a file easily in CVS was quite an annoyance but nothing that made it completely useless.
There were always workarounds. But exactly those areas were workarounds or better tooling patches over stuff Git doesn't do are the areas that a possible successor will address.
"None of the other big VCS had fatal flaws. If they had fatal flaws, they wouldn't have been used."
That's just quibbling about definitions in fuzzy English. If you want to insist they weren't "fatally flawed" at the time, fine, but they certainly are relative to 2020 even by your definition.
You're really doubling down on applying deliberately hostile definitions to what I said, aren't you? I get to tell you what I meant. You don't get to pick the definitions I was using, especially after I told you you explicitly you picked the wrong one.
You didn't really define what you meant with "fatal flaws". You just claimed that git didn't have them.
But if your Subversion example is one of those fatal flaws, then I just link to his example of Git usage that probably happens more often than we want to admit https://xkcd.com/1597/
Using your tool wrong is not a fatal flaw of the tool.
These fell by the way side as they never had consensus of being better than SCCS. Now git has the weight of the Linux kernel behind it which pretty much EOL'd all other source control mechanisms.
I don’t think Linux kernel matters that much - only a tiny fraction of the total number of developers use it.
From what I saw, it was git vs hg - which had different philosophies. Hg gave you nice, polished workflow for supported tasks. Git gave you building parts that you can make your own system from. It turned out that enough programmers wanted to build from parts.
I actually prefer SVN and HG style branches where you keep the branch history as a known formal branch, then have one merge back to Trunk.
It is unclear to me how far back you are saying SVN did not have branching.
I don't remember a time when Subversion didn't support branches. I'm sure Subversion 1.0 supported branches (although I can't find evidence of that right now, but I also can't find any release notes for Subversion 1.n containing "now we support branches!")
One can argue about what a branch is, for sure. Subversion branches, Mercurial branches and Git branches all have very different implementations.
But the main workflow of creating a feature branch, developing a feature, then merging it back into the trunk/master, has, as far as I know, always been supported in Subversion.
> I don't remember a time when Subversion didn't support branches.
SVN's official response to branching and tagging was to copy directories around in the repository. Arguably that means it does not support branching, at least according to the concept that has been in place for the last decade.
That's true that there was no command "svn branch", there was only the command "svn copy". It's true that command could have been better named, like many Git commands could have been better named.
However, don't let that confuse you. The "svn copy" command creates a new place where you can do independent development, and where afterwards merging back can occur. So it's the same as a branch by that definition.
An "svn copy" is a lightweight copy, with a reference back to the place the copy was made from.
- "svn merge" does its merge by looking at what commits have been made at the original location (since the copy), and the commits to the copy.
- "git merge" does its merge by looking at what commits have been made on the original branch (since the new branch was created) and the commits on the new branch.
So "svn merge" and "git merge" act in the same manner. (Of course there are differences in the algorithm, but I wish to refute the point that Subversion "does not support branching [at all]".)
And if you never use "svn merge" or "git merge", then a branch is just a copy, in either system.
> That's true that there was no command "svn branch", there was only the command "svn copy". It's true that command could have been better named, like many Git commands could have been better named.
It was not a naming issue. It was a bona fide lack of support for a basic feature. The manual itself states quite clearly that in SVN land you create a branch by copying around the entire working directory within your repository. SVN's manual is quite clear on how SVN actually tracks the state of a file system, and not the state of a source code tree. By copying the working directory around the repository you're creating new revisions of your file system. That' it. Just because you can diff two directories it does not mean copying a directory around in a file system is a branch. And nor is it a tag.
Considering CVS had branches, Subversion probably had them from the start or at least some early development version. Very different under the hood though, as you say.
Not sure what variant of SVN you were using that didn't support branching, everyone I knew used branches.
It did have problems with occasional inexplicable tree conflicts, which I do not miss. But that's the same as the pain of trying to rebase in git when things have diverged and it just starts spewing repeated conflicts for each and every commit. In either case, the easiest thing to do is make a fresh branch, patch your changes over, and go from there vs. trying to reintegrate the broken branch.
Subervsion did support branching but it was so fragile (especially before version 1.5 or 1.6 I don't remember) that some went out of the way just to avoid using branches in SVN. Those dreaded tree conflicts…
One big thing that DVCS brought to the table were local serverless repos. Now you could version all the things, with very little effort, and promote it to a "proper" repo later.
The other big thing was free cloud hosting for the repos.
Neither of these are unique to Git, obviously, or even introduced by it. It just happened to be the right combination of features and speed at the right time to become the winner of the popularity contest. Kinda like C.
And, just as C is still around, for all its numerous horrible quirks, Git will likely be around for a long time as well.
Every version control system that's become dominant in my lifetime became popular because it fixed a major obvious flaw in the previous dominant system (RCS, CVS, SVN).
From where I sit, Git has a couple obvious flaws, and I expect its successor will be the one that fixes one of them. The most obvious (and probably easiest) is the monorepo/polyrepo dichotomy.
Can you explain more about that? I've worked with both and I feel like monorepo is kinda a pain, but I dont understand where Git fits in to either directly. Seems like it just snapshots files.
In this context "monorepo" means a huge repository with many many revisions. Git has several well documented deficiencies in this scenario. See [1] for microsoft's experiences with git.
It's only a problem at a really large scale. At the scale of Microsoft or Facebook, there are factors that lead to the use of a monorepo being more efficient. At that big of a scale, companies have enough resource to develop internal tooling to deal with the problem (e.g. the use of Mercurial at Facebook).
It would be nice if you and everyone else stopped gatekeeping this problem.
We have 600 devs and face these problems. I can assure you we sure as hell dont have the resources spare to reroll git. We're way too busy rerolling everything else.
FWIW, in case of Microsoft at least, it's more a question of product size than company size. Microsoft doesn't use a single monorepo for everything, like Google (so far as I know) does - just look at http://github.com/microsoft/; and that's not even counting all the VSO repos! It uses product-specific monorepos for some large products.
Merge conflicts are not so scary, and are an elegant way to handle distributed changes with simultaneous edits to a single file.
If the conflict is huge, rebasing can help you by "playing" the commits from one branch one at a time so the conflicts are smaller / easier to fix.
At a previous job, a team was forced to use checkout-style VCS due to their manager's unfounded fear of merge conflicts; I couldn't go in that office without hearing one developer shout to another: "Hey, can you finish up and check in that file so I can get started on my changes?"
I’ve spent too much time helping others fix bad merges, and I still catch myself making mistakes. There’s a lot of work that could be done for clarity and error avoidance.
However, in some cases (like non-mergeable binary files), it is actually better to have a system that allows one user to take a lock on a file and have exclusive editing abilities. The git protocol has no support for those workflows, and so people end up using a Google Doc or something to track who is modifying what file. Definately a place for improvement.
This consideration is actually irrelevant to locking non-mergeable binary files. It doesn't matter what branch we're on or where the file is located, only that you and I both want to edit the logo. Eventually, either your version must be based on mine, or mine based on yours, since they will be merged.
So it's probably better not to have that file in Git, since it doesn't support the workflow around which Git is based.
It's actually right to store your design documents in Google Docs or a wiki and your code in Git, rather than everything in Git.
It is easy to have one filestore to rule them all and in the darkness bind them, but if you want to do different things with them, you have to do different things with them. I'm not sure that it's possible to unify text file and binary doc based workflows, but it seems we don't have to worry because users automatically use the best tool for the job and it's only hackers who tie themselves in knots trying to make git do everything.
The only way to make that simple is to centralize the version control system so that you can have a single arbiter of who has what locked. To add easy locking to Git, you'd have to turn it back into a non-distributed VCS.
I don't think I need to sell the value of DVCS over VCS, but what seems to get lost is that buys you a certain amount of essential complexity, expressed in the CAP theorem and its consequences.
I'm talking about purely advisory locks. Accessing a locked file would let you know who locked it, and if you unlock it, it would just notify them. So it's just a communications mechanism in addition to regular merges.
The alternate method is that locking a file marks you as an interested party to a merge, allowing you to review the correctness of a merge.
This would be purely to avoid changes being lost during merges.
I've worked in teams where developers seemed terrified of merge conflicts, to the point of telling eachother not to edit a particular file which seems absurd. Maybe I don't know any better but they seem like part of life.
When at least one person is making changes that touch large parts of the file, it's very sensible to ask others to not touch it, if you don't want to spend hours merging it manually later.
Large files and long histories hinder its total dominance in the game and art industries. Because of git's shortcomings polyrepo is a near necessity not simply a stylistic choice. LFS is a bolt on solution that could/should have better support.
> Because of git's shortcomings polyrepo is a near necessity
I'm intrigued by this claim. I've come to the opposite conclusion - that monorepo is near necessity with git because there's no tools for branching/rebasing multiple repos at once.
After using both I can say both have problems. Polyrepos lack tools for working with multiple repos simultaneously, and require more attention to versioning. Monorepos have longer histories, and the large number of objects can hurt performance.
Which one you should use depends on which downsides are less impactful for your use case.
Git stores the diffs in chronological order doesn’t it? I recall reading about someone doing a commercial implementation where the commits are stored in reverse chronological order. I’d been thinking that was github but I’ve never been able to find the article again.
Git's model (which it copied from monotone IIRC) is not diff-based, it's snapshot-based. That is, commits are not stored as a diff to the previous commit, but as the whole state of the tree plus a pointer to the previous commit(s).
As an optimization, when it packs several objects together in a pack file, it can store objects as a delta to other (possibly unrelated) objects; there's a whole set of heuristics used to choose which objects to delta against, like having the same file name. And yes, one of these heuristics does have an effect similar to "reverse chronological order"; see https://github.com/git/git/blob/master/Documentation/technic... for the details.
> It would also be nice to have a repo that isn't language-agnostic. It's too easy to track non-semantic changes, like white space.
In my opinion this is a problem with programming languages rather than version control. Namely we mix presentation and representation when using text as our source code. In the case of whitespace we have an infinite number of syntactic presentations which all correspond to the same semantic representation. Tooling has been created to try to deal with this such as code formatters which canonicalize the syntactic presentation for other tools. Git itself even has to deal with this because of platform differences, i.e. LF and CRLF.
I don't think that's true. I've been using git's CLI since I started using git a few years ago, and exactly zero of my problems with git could've been solved by a different user interface (be it GUI or a "better" designed CLI). Pretty much all of my problems have been with my lacking understanding of the abstractions that git uses to make all of the powerful things it can do possible.
You are in the minority. It's such a ubiquitous experience, a running joke in the industry. Saying a tool is useful and powerful, is fine and good. That sentiment has nothing to do with usability.
Isn't that the same thing? If you need to be taught the underlying abstraction to be able to understand the UI you've got a text book example of leaky abstraction.
After I had to unfuck a repository for the n-th time, I trialed a switch to Mercurial, we switched shortly after. I can count on one hand how many times I've had to intervene in the last few years.
I tend to think that it's fine, because I remember what it used to be like... what we have now is the "easy" UI!
And they do occasionally put some new stuff in that helps. Like the recent version which adds new commands to split out the two completely different uses for `git checkout` (making/switching branches and reverting files).
There is zero doubt in my mind that it's Github that "made" Git. Without it, it would be just another one of many DVCSes. Git's value isn't inherent, it's all down to network effects.
yep, nemo got it right. git basically hacks cherry-picks in the same way previous VCS’s hacked branches.
unfortunately there are no patch-theory based VCS’s with a practical level of usability. what git was to monotone, X is to darcs/pijul, where X hasn’t been created yet.
How about history tracking for file renames doesn't work well? It's hit and miss if git blame --follow works.
Subversion did this better even before Git existed.
Git is completely oblivious to moving code from one file to another. Git blame will never show you the original commit if you just relocated a method to another file. Due to this, refactoring often put additional hurdles into exploring the code history.
Git submodules have lots of problems. I wanted it to work like a symbolic link to another repository so that I could develop both at the same time on the same machine. Just like how pip allows me to install a Python package in editable mode. If I make a change in the submodule, the superproject should automatically see it.
If I were to pie in the sky dream up a replacement for git, I'd have it store the AST of the parsed code instead of a text file. It would solve a lot of problems with refactoring crapping all over the history. Like I said, pie in the sky. Probably never gonna happen.
Personally I don't see git's problems with large binary files and tens of millions of commits as being major issues. Those two alone are way less valuable than git's ecosystem and mindshare.
It would have to be specific to certain languages, which would, in turn, hinder adoption of new languages to some degree if the git-next took off. So, I'd prefer not to have that be a feature. :)
I think you could write it as the ability to a diff on a binary ast without fussing too much about what the ast represents. Then you merely need to write a parser/serialiser combo for your language to the ast as a repo plugin.
Otherwise it won't just be new languages which suffer, but users of supported languages will suffer when there's an upgrade.
What you want has nothing to do with git (which is a storage model). You can use arbitrary diff and merge resolution algorithms with git's plumbing, which would give you the AST-aware functionality that you want.
That's like saying DVCS has nothing to do with VCS -- it's just the server model. Every major advance has been accomplished by increasing the scope of version control.
Arbitrary diff/merge in Git is a great example of the Turing Tar-Pit. It's possible, but prohibitively inefficient for many things I want to do. You can't add your own types, index, or query optimizations.
Today, if I want to store data for my application, I have a choice between good support for rich object types and connections (e.g., Postgres), or good support for history and merging (e.g., Git). There's no one system that provides both.
Couldn't you store the exported database as sql commands? I'm not familiar with every git hook, but if there aren't enough to automated that I guess you could wrap it.
The slowness of destroying a whole database and then recreate it when checking out should be something you can handle by relying on the diff to generate a series of delete commands and a series of insert commands.
But yeah, I guess committing will be slow if you have a lot of data to export. For the time being, it's a trade off to be made.
[I might consider testing this with my current database project. But I'm using SQLite so I guess that implies a lot less data than Postgres.]
DVCS indeed has nothing to do with VCS, it has a lot to do with the data model used by the VCS.
A modern but still centralized VCS like Subversion or Perforce is what you get if you first add networking (CVS) and then atomic commits. Without atomic commits you are pretty much forced to keep a centralized server, and Subversion didn't try to change the server model after adding atomic commits.
DVCS instead is what you get if you start with local revision tracking like RCS, and add atomic commits before networking. Now the network protocol can work at the commit level and is much more amenable to distributed development.
frutiger is correct that the diff algorithm has nothing to do with git itself, in that git can accept pretty arbitrary diff algorithms in the first place for all the commands that take one.
Check out git-diff(1) and --diff-algorithm. --anchored is the one I find the neatest.
> Today, if I want to store data for my application, I have a choice between good support for rich object types and connections (e.g., Postgres), or good support for history and merging (e.g., Git). There's no one system that provides both.
I like the way you put this. In case anyone's interested in brainstorming I'm dabbling in this problem with a thing called TreeBase (https://jtree.treenotation.org/treeBase/). It's still a toy at this point, but it stores richly typed data as plain text files to leverage git for history and merging and then can use SQLite (or others) for querying and analysis. A very simple database in the wild looks like this: https://github.com/treenotation/jtree/tree/master/treeBase/p...
Why do we have byte code? Why not run everything in interpreters? Because parsing pure text takes a lot of work. So we store it in an intermediate mode to economize.
Saying just parse it every time is denying that there are very real costs associated with hat decision.
I used VisualAge's Envy for 3 years while working in a Smalltalk project.
Envy does versioning of classes and methods, and you can programmatically access to the model.
It allowed us to build tools around the VCS. For example, we had tools to merge multiple feature branches and resolve conflicts automatically.
We also used the same tools to produce migration scripts for our database (GemStone).
That was 18yrs ago! and today sounds irreal.
You can build tools on top of git, but the versioning “unit” gets in the way. (e.g imagine the possibility to encode refactorings in your change history and reapply or rollback them).
I’m not trying to criticize git. I think it is the best file based DVCS. My point is that many dev tools that we use today are extremely rudimentary, because the lack of good abstractions. And I don’t think that git provides a good model to build those abstractions on top of it.
I want an editing environment which operates on the AST of my code (obviously it would have to support every language I wanted explicitly to do this), so that files become entirely irrelevant, I never have to worry about formatting differences or where things are or whether that function is in that file or that file. A bit like working in a Smalltalk image.
If that was then extended into the version control system that'd be even better. Oh yes.
But getting a new language into these things would probably be a nightmare.
Look into Unison, a language that stores the AST and immutable history of all functions to provide a combination of package manager, IDE, and DVCS. Once you store the AST and all history, some fascinating side effects happen!
Most language ASTs don't encode unfinished or work-in-progress code very well (the difference in AST shape between missing one `{` and the fixed code can be substantial). You may think it better to always only commit working code, but your source control system is also a backup system if you need to save a work in progress branch to come back to it, and also sometimes a communications system if you want to request a coworker examine your code to help you pinpoint bugs you can't find or review work in progress.
Most language ASTs also don't encode useful-to-the-programmer but useless-to-the-compiler information like comments and whitespace. There's been good progress in that (the Roslyn AST system has some neat features), but in practice an AST is always intended more for the compiler than the user/source writer. This also is reflected often in speed, a lot of languages have a relatively slow AST generation (which would add sometimes very noticeable wall clock time to commit time, depending of course on language and hardware).
Plus, of course, all the usual bits that ASTs are extremely varied among themselves (some are weirder DAG shapes than trees, for instance).
An experiment I ran was to the use the "next step down" from full AST which is your basic tokenizer / syntax highlighter. Those designed to deal well with malformed/unfinished/work-in-progress input, and to do it very quickly. Years back I built a simple example diff tool that can do token-based diffs for any language Python's commonly used syntax highlighter Pygments supports. [1] In my experiments it created some really nice character-based diffs that seemed "smart" like you might want from an AST-like approach but just by doing the dumb thing of aligning diff changes to syntax highlighting token boundaries.
You could even use it/something like it/something based on it today as your diff tool in git if you wanted, with the hardest part configuring it for which language to use for what file. (I never did do that though, partly because the DVCS I experimented with this for didn't have a pluggable diff system like git does, nor did it support character based unidiff as a storage format which the experiment was partly to prove both ideas could be useful.)
I’ve read a couple descriptions of the internals of git that don’t disagree with the design of SVN, so I’m not unsure why you can’t theoretically check out a single subtree. Either the documentation is too hand wavy or some implementation details have blocked that possibility.
It is necessary but not sufficient for any new contender to do at least the following to have any chance of taking over:
- Interoperate with the major player(s), currently Git and in many places unfortunately still Subversion. svn2git probably did more for Git adoption than any other feature or tool, because it allowed a fairly painless transition without losing information.
- Solve at least one big problem with the current contenders. Git made it possible to run VCS without a separate server program and sped up VCS operations massively. Both of those were huge. Looking at the Fossil home page[1] it does have some features I personally have wanted in VCSes, such as integrated "bug tracking, wiki, forum, and technotes," but the devil is very much in the details of how that actually works (How easy is it to write your own custom frontend or add business-critical bug tracking fields, for example?), and it's not like we don't already have good bug trackers, wikis etc.
Just as a tangent, some issues (but not necessarily major or fundamental, depending on who you ask) with Git as it works right now:
- Does not use cryptographically secure hashes, and has no clear migration path to a different hashing mechanism.
- Git Annex is not yet built in.
- The command line is complex, including many niche subcommands and tons of rarely used options.
- The command line is inconsistent, such as `git rm` vs `git branch --delete` vs `git remote remove`.
- It is based on a less than ideal theoretical model of patches[2]. IMO this is the most exciting development in VCSes since Git.
Everyone's has their own tastes and preferences, of course, and I respect that yours is different than mine. That said, I used and loved CVS and then SVN for years and didn't get why all the kids were fussing around with this new Git thing. I finally made myself try it for about a week. At the end of that experiment, I ported all my repos from SVN to Git and quickly set to purging all Subversion-related knowledge from my brain. There's literally nothing about SVN that I prefer to Git, other than its UI was a little more pleasant.
I actually LIKED having a central repository, which many of us still seem to prefer (i.e. GitHub, GitLab, Bitbucket). I switched to git mainly because my colleagues were all using it. I found it difficult to use, at first, because of my expectation of a central repo. Many years in, however, I see extreme value in having all your history locally. Specifically, never having to worry about a server crashing or your "host" going out of business.
Same. I didn't get what was better about a distributed system for small projects, until it fully clicked with me that my copy was just as "official" as any other sitting around, and that any "server" was a copy that we collectively decided was going to be the one of record. I can't imagine any plausible scenario in which I'd go back to having a central SVN-style server.
Having an official central server is precisely what you want in most development environments. Being able to go back to any prior version of the code is tremendously useful and something git can't do.
Having history available locally also means you can perform interesting operations on history -- like "git blame" -- without making the server do all the heavy lifting.
Right, especially after a "squash", which seems to be the standard way to merge branches in the companies I've been working with recently. (Which is, ironically, also the way Subversion merges branches. With the exception that "svn blame -g" will go into the commits which were squashed if you want. An option which doesn't exist after a "squash" on Git.)
Fossil's opinion on this is that history is an immutable record of project history. It may be messy and unfortunate at times, but it is what happened, and it shouldn't be altered in place any more than you'd do that with an accounts ledger.
If you are their superior, other users disregarding your orders is s social problem, not a technical one. If you aren't, it's a good thing they are able not to do what you want.
Tools being more flexible is strictly a good thing. If they are misused, the person that misused them is responsible. It is that simple.
> I actually LIKED having a central repository, which many of us still seem to prefer (i.e. GitHub, GitLab, Bitbucket).
You're confusing a hosting service with being forced to use a centralized repository.
Take GitHub, for example. If git was centralized them you would not have forks, multiple remotes or multihosting, or could even work independently of the remote server.
With Git, you can even set up a repo in a network file system somewhere, or even a USB thumbrive.
Git makes it easier to do a fork but you could certainly fork a subversion repo. I’m fairly certain you could sort of a merge process across multiple upstreams.
The "fork" concept is not native to neither Subversion nor git.
You are probably thinking of branches and tags, and those are used similarly in both systems. They are a bit more convenient in git since they are created in constant as opposed to linear time.
Copies in Subversion are only metadata. Since partial clones are native to the system, it is simple to present both branches and tags as file paths. It was likely considered an easy user interface. Everything is a file and all that. In comparison, git users must learn the git object naming scheme, otherwise things can end up very confusing should you have a directory and a branch with the same name.
No, I was referring to forking projects, in the sense that Git allows for creating brand new and independent repositories that clone the version history up to a point in time and enable to add the origin Git repository as a remote repository.
SVN also does not support branching or tagging, as it actually supports only copying directories around a file system.
For most of us, even with DVCS there is one copy of the repos that is more important than all the rest - the one attached to the CI or deployment tools. It’s only decentralized as long as this copy has only short periods of unavailability. Hours are bad, days are much worse.
Linus seems to have a very different development model than almost all of the rest of us. And if Linus is asynchronous, then so is the codebase he maintains.
If you or I have eight different PRs going it’s because we have eight different promises being made to customers. Rejecting them has real consequences that we feel. For Linux many of those are externalities.
> Which is a better choice than Git for most projects, to be honest.
That's pretty subjective. Most engineers I interact with personally know how to use Git but not SVN. I'd argue the opposite is true base on my subjective experience.
Really depends on where/how you're using versioning, team familiarity, etc.
It took me maybe eight months before I was willing to perform open heart surgery on a subversion repo. Two years to dare to try the same thing with git, and I was much more anxious about the whole thing. At the three year mark I have the same ambivalence about things working out that I did in less than a quarter of the time with the previous system.
That’s a little too much to explain away with variability.
With time, the standard requirements rise and we get more demanding.
How about a VCS that not only diffs line by line but saves the diffs as editing delta?
Or that understands the syntax of the code and can differentiate a variable renaming from an actual code change?
Or connecting different repositories is hardly solved with Git. Submodules suck, grafting even more so. Or you can't make one repository out of two and keep their Git history intact.
I’ve been wondering if we need a slightly more general tool for tracking changes, that happens to have a VCS commandline as a first class client of that system. The problem is how to do bindings in many languages, or share one binding with some sort of IPC/RPC protocol.
I have some collaboration tools I’d like to write but creating my own edit histories and conflict resolution is daunting.
Ammending changes the hash of the commit. That doesn't matter if you haven't pushed it yet but if you have, you can't push the change upstream without force-pushing.
And if I want to change a commit message further back in history (regardless of my own or of somebody else), it would branch off at that point from the upstream git history.
What I want is that you could put multiple comments onto the same commit with different timestamps. Like git versions files, it should also allow to version commit messages.
You can accomplish something like this by using git-notes (https://git-scm.com/docs/git-notes). You'd have to have your own tooling to read/write these in a convenient way of course.
Yes but they are just a patch over the missing functionality.
The key point is "your own tooling". Git has a great tool universe but unfortunately you immediately lose many tools if you go beyond what the core offers.
Not necessarily. Fossil's `amend` command works by adding additional information to the repo that the web UI and commands like `fossil info` look at when building up information intended for direct consumption by the user.
In this way, you can edit commit messages, rename branches/tags, add/remove tags, etc. to historical check-ins without breaking the blockchain / Merkle tree of commits.
This allows Fossil to keep all of the historical information about what happened to a given file, commit, ticket, etc. while still allowing a coherent presentation of the current state of affairs to the user.
Of course Git is replaceable. When a new tool that provides a substantial benefit over using Git comes around, it will be replaced.
If that never happens, then it's a pretty strong indication that Git is working perfectly fine -- so why worry about whether or not you can replace it?
It feels that way, and it's heartbreaking that our entire species ended up locked into a tool with such a horrible interface.
In a way, the git monopoly is worse than Windows or x86 or IPv4, because it's not just a piece of technical infrastructure. Its arcane commands and its branching model have infected all of our brains. You can choose a different editor, you can choose a different operating system, but for as long as we all live, we will never escape the fact that "git reset" does a half a dozen confusingly different operations, or that renaming cannot be tracked, or that most users don't fully understand most of the commands they regularly use.
I'm working on a git replacement, in a way. The thing that makes git powerful is that its just text. Git is probably the most powerful thing for code as text. When code no longer is just text (and by "text" I mean bytes on disk, not that we're switching to coding with emoji or VR) you get to do more powerful stuff.
Our plan in Dark (https://darklang.com) is to combine all the different ways that people "branch" (deployment, feature flags, git branches, staging/dev/prod environments) into a single concept. And then we also plan to combine all the ways to "comment" (PRs, slack messages, commit message, code comments) into a single concept.
Not sure if you'd call that a git replacement, but it's a displacement of sorts - the function of git is replaced by non-git.
The second one is just an idea right now, suggestions welcome. The observation is that comments on a particular line of code are spread in as many as a dozen places (a google doc, slack, trello, the code itself, an old version of the same code, PRs on github, comments on commits on github, commit messages, another place in the codebase referencing this one, the docs folder in your repo, another repo that uses this API, your 3rdparty docs on README.io, etc). This is weird and bad, and it must be possible to do better.
For comments, fully agree that we don't need several isolated systems (source files, docs, commit messages, ...)
How about one system which is a hypermedia system and can hold many different kind of objects. So annotation objects can reference 'code objects' such as types, fields, functions or even blocks directly. Not sure if Dark gives each of these an identity (it should) which will make it possible to refer to them via hard links rather than text snippets. Once you have all code and comment objects in one hypermedia system, creating views from that is about multiple projections of the interconnected objects.
Wouldn't it be great if I can use a query to refer to 'all functions that reference this type' inside some docs? Or list all annotations that reference a function? These could be embedded inside annotations as well. Gtoolkit does something similar.
Perhaps even a 'branch' can be thought of as a subset of the hypermedia graph. E.g. using a code block X2 instead of a code block X1 within the same function.
> 1. Metcalfe's original Ethernet has been replaced a bunch of times...
These replacements were seemless to users. New Ethernet adapters were compatible with at least the previous spec. The Git import/export of Fossil is not seemless at all. It actually adds quite a bit of complexity if you want to introduce it your regular workflow.
> 2. Microsoft's long-term stalwarts Windows and Office are dying...
Citation needed.
> 3. Adobe's having a hard time hanging onto its old market...
Citation needed.
> 4. IPv4 still won't go away...
There is a lot of hardware out there that only works with IPv4. The costs and risks of switching your org's internals, product or services from IPv4 to IPv6 are phenomenally higher than switching your org from Git to Fossil or adding Fossil support to your product or service.
If Fossil is truly superior to Git and people are not switching to it, then there's no hope they'll switch to IPv6. Not until the cost of not switching is greater.
Wifi mostly replaced ethernet for user facing applications.
There was partial protocol compatibility between ethernet and wifi, but the user experience was very different. Some features were lost or degraded (eg speed, reliability, security, configuration complexity) but a pain point (cables) was fixed.
I really doubt Git is going anywhere for the foreseeable future, but I could imagine a more approachable VCS catching on. Git is extremely opaque to most new developers and even for experienced devs looking up a new command. Sometimes I look up how to perform an unfamiliar task with Git and find 4-5 competing answers on Stackoverflow with no real clear explanation of why one is better than another.
If an easier VCS caught on enough to be used in schools and boot camps, many younger devs would start with that one and just keep using it as they progress in their career. The new VCS wouldn't have to be better or more powerful, just easier to use for the basic stuff that makes up 95% of Git usage, in order to gain traction. That said, Git works perfectly fine and is so entrenched that I don't see it going anywhere.
Git's UI is unbelievably bad. However, it's practically impossible to avoid learning how to use it these days, so "much better UI than git" will never be a compelling selling point for an alternative VCS: almost its entire target market has already paid the cost of learning git.
That problem could be gotten around if there was some enormous pool of potential VCS users who aren't using VCS currently but would if there was a better one. I don't think there is.
The obvious pool of people who don't currently use VCS's is everybody who mostly deals with files that aren't plain text. Using git to collaborate on Photoshop documents, Word docs, Jupyter notebooks, videos, or any other non-plain-text format is a frustrating nightmare, and a VCS solution that provided revision tracking and collaboration for all those filetypes would open up a new set of users.
However, it's practically impossible to avoid learning how to use it these days, so "much better UI than git" will never be a compelling selling point for an alternative VCS
I think this is true if you think the target market for VCSs is professional software developers only. But there are other people writing code that might care less about the fact that Git is the standard for professional software development and don't want to pay the cost of learning or using Git: data scientists, scientists in general, new media artists, most high school students, etc. I've had a hard time selling Gitless (https://gitless.com) to undergrad CS students, but it is easy to sell to non-CS students that write code.
That said, GitHub is a big thing so any new VCS probably needs some story for Git-compatibility. Even tools that have built-in version control like Overleaf have some way of synchronizing with Git repos.
Thanks to Github, git is the standard for non-professional software development as well. My kids are in high school and they're using it.
Gitless looks great. It looks like you nailed all the main issues I have with git. If you ensure that people never have to use the regular git interface, maybe it'll take off. I hope so!
Thanks to Github, git is the standard for non-professional software development as well. My kids are in high school and they're using it.
Yes, but maybe they care less about using Git compared to another VCS as long as the other VCS interacts with Git and they can put their repo on GitHub. The question is:
If you can use a VCS that is easier to learn/use than Git and that is compatible with Git so that you can put your repo on GitHub if you want to, would you use it? If no, why?
My guess is that most professional programmers would answer "No, because Git is an industry standard and I need to know Git to get a good job", while other people that write code but have no intention on becoming professional programmers are much more likely to answer "Sure, why not!".
If you ask your kids, I'd love to know what they said :)
Gitless looks great. It looks like you nailed all the main issues I have with git. If you ensure that people never have to use the regular git interface, maybe it'll take off. I hope so!
It doesn't have a UI. Its just a program that takes instructions and does what you tell it. If you want a nice fancy GUI for Git there are plenty of reasonable options. Fork is the one my coworkers seem to be enamoured with at this point in time. I myself don't see the need to use any sort of GUI for Git the vast majority of the time.
A better Git UI is possible, like Gitless. A GUI that just has the functionality of the Git CLI doesn't solve the problems I care about, like the staging area being unnecessary and complex.
Bob Martin claims at various points in time that half of all developers have less than five years experience, and attrition and expansion of CS degrees has maintained this.
Five years after introducing a replacement you could have half a team that never used it at all, just like SVN is now.
Yes. I've been programming for nearly 40 years and I only had to use Subversion a couple of times; I never really learned it. OTOH I have to use git for all kinds of projects. Learning git is essential in a way that SVN never was.
I think part of the reason is that SVN never took over from CVS as completely as git took over from everything else. I think another part of the reason is that the number, size and scope of open-source projects exploded since git and Github appeared, and they mostly chose git.
Also relevant: the only project that I recently interacted with that was using SVN is LLVM, and it was faster/easier to use the Github mirror than to use SVN.
I wonder how possible it is to retain the Git data model, but completely replace the CLI? Git gets some flak for its conceptual decisions (e.g. what "branch" means), but IMO most of the real-world friction comes from numerous little inconsistencies and general weirdness.
Git's biggest flaw is that it doesn't scale. If a new system can fix that without sacrificing any of Git's benefits, I think it can topple Git.
It's ironic that Git was popularized in the same era as monorepos, yet Git is a poor fit for monorepos. There have been some attempts to work around this. Google's `repo` command is a wrapper around Git that treats a set of smaller repos like one big one, but it's a (very) leaky abstraction. Microsoft's GVFS is a promising attempt to truly scale Git to giant repos, but it's developed as an addon rather than a core part of Git, and so far it only works on Windows (with macOS support in development). GVFS arguably has the potential to become an ubiquitous part of the Git experience, someday... but it probably won't.
Git also has trouble with large files. The situation is better these days, as most people have seemingly standardized on git-lfs (over its older competitor git-annex), and it works pretty well. Nevertheless, it feels like a hack that "large" files have to be managed using a completely different system from normal files, one which (again) is not a core part of Git.
There exist version control systems that do scale well to large repos and large files, but all the ones I've heard of have other disadvantages compared to Git. For example, they're not decentralized, or they're not as lightning-fast as Git is in smaller repos, or they're harder to use. That's why I think there's room for a future competitor!
(Fossil is not that competitor. From what I've heard, it neither scales well nor matches Git in performance for small repos, unfortunately.)
I'd say the number of git repos on Earth that would encounter problems of that nature would be a vanishingly microscopic minority. Sure, it's a problem for those companies but it's not a problem for anyone else.
All of the organizations that have outgrown git will have such incredibly specific requirements meaning nothing but a custom built tool will work for them.
Let's say you started with a well factored set of code that is managed within your organization. What advantage is there to having multiple repos if you're not limited by your tools? Refactoring is easier within a single repo...
There probably will be a plethora of different hard issues to fix in such situations.
It's also easier to institute change in a dictatorship as opposed to a democracy (being a dictator that is :).
In my experience, code doesn't stay well factored unless there are technical hurdles that keep it so. That of course doesn't have to be a repo boundary, but in can be.
This reads to me as a failure of imagination. Any mid size game development shop is going to feel this pain - not just giants like Microsoft and Google. I believe the Unity Game Engine has a user base in the millions? Even a subset of that may be small in comparison to the entire developer population but by no means microscopic.
I disagree that Git's biggest flaw is its lack of scalability. Cases where git needs to scale tend to be isolated to companies that have the manpower to build a finely-tuned replacement (see: MS, Google).
Git's flaws are primarily in usability/UX. But I think for its purpose, functionality is far more important than a perfect UX. I'm perfectly happy knowing I might have to Google how to do something in Git as long as I can feel confident that Git will have the power to do whatever it is I'm trying to do. A competitor would need to do what git does as well as git does it, with a UX that is not just marginally better but categorically better, to unseat git. (Marginally better isn't strong enough to overcome incumbent use cases)
And for the record: I think git-lfs issues are primarily usability issues, and tech improvements. The tech enhancements will be solved if there's enough desire, and as I mentioned the usability problems are more annoyances than actual problems.
Scaling isn't even just about number of files or size of them. A problem I've hit is just in having cross-repo stuff work well. Monorepos are helpful partly because git submodules are not ideal for good workflows, and splitting stuff across multiple git repos can backfire (it doesn't help that almost all the tooling around CI and the like is repo-based instead of project based).
I would love a layer over Git to handle workflow issues related to multi-repo projects
> Cases where git needs to scale tend to be isolated to companies that have the manpower to build a finely-tuned replacement (see: MS, Google).
I constantly run into git scalability issues as an individual. I don't use any of the UI clients because they all fail hard on mostly-code git repositories. I abandoned my VisualRust port in part because the mere 100MB of mingw binaries involved for that meant it was using github LFS, which meant CI was hitting github quota limits, and as I wasn't part of the organization - nevermind an admin with billing rights - I couldn't even pay to up said quota limits paying out of pocket myself even if I wanted to.
I'm not going to inflict git's command line experience - which confounds and confuses even seasoned programmers - on any of the less technical artists that might be employed at a typical gamedev shop, even if git might be able to scale acceptably if locally self-hosted at a single-digit employee shop.
A few dozen or hundred employees? Forget it. Use perforce, even though it costs $$$, is far from perfect, and also has plenty of scaling issues eventually.
The whole reason git lfs exists is to workaround git scalability problems. Its raison d'etre is problems with git.
That one of - if not the - most popular tool to solve said git scalability problems, also has scalability problems in practice, is both ironic - and absolutely a problem with the git ecosystem. To be pithy - "Even the workarounds don't work."
"Technically", you might say, "that specific symptom with git lfs, and that service provider, isn't the fault of git the command line tool, nor the git protocol". And you would be technically correct - which is the best kind of correct.
But I don't think we're referring to either of those particularly specific things with "Git" when we ask the article's question of "Is Git Irreplacable?". I'm already the weirdo for using git the command line tool - most of my peers use alternative git UI clients, and I don't mean gitk. The git protocol is routinely eschewed in favor of zips or tarballs over HTTPS, Dropbox, Sneakernet, you name it - and is invisible enough to not be worth complaining about to pretty much every developer who isn't actively working on the backend of a git client or server. Not to mention it's been extended/replaced with incremental improvements over the years already.
So I'm using a slightly broader definition of "git", inclusive of the wider ecosystem, that allows me to credit it for the alternative UI clients that do exist, rather than laughing off the question at face value - as something that has already been replaced.
Not waiting to provide you with free storage is not a "scalability problem". I can't spend company money on Perforce either, is that a Perforce problem?
I pay for a github subscription. I set out to pay more for a github quota bump, but found I was limited by upstream's LFS quota rather than my fork's LFS quota.
Nothing about your problems had anything to do with git & everything to do with the commercial service you were using for your source code hosting.
Github the company is not interested in providing you (or anyone else) with free storage for arbitrary data. You were unable to pay for the storage options they do provide because you did not have admin rights to the github account you wanted to work with.
None of this is a problem with git, be it GUI git clients or command line ones.
This isn’t just "technically correct". It’s the "a commercial company doesn’t have to provide you with a service if they don’t want to" kind of correct.
> Nothing about your problems had anything to do with git & everything to do with the commercial service you were using for your source code hosting.
All the commercial service providers recommend keeping total repository sizes <1GB or so, and I hear nothing but performance complaints and how much they miss perforce from those who foolishly exceed those limits, even when self hosting on solid hardware - which is 100% the fault, or at least limitation, of git - I believe you'll agree.
LFS is a suggested alternative by several commercial service providers, not just one, and seems to be one of the least horrible options with git. You're certainly not suggesting any better alternatives, and I really wish you would, because I would love for them to exist. This results in a second auth system on top of my regular git credentials, recentralization that defeats most of the point of using a DVCS in the first place, and requires a second set of parallel commands to learn, use, and remember. I got tired enough of explaining to others why you have a broken checkout when you clone an LFS repository before installing the LFS extension, that I wrote a FAQ entry somewhere that I could link people. If you don't think these are problems with "git", we must simply agree to disagree, for there will be no reconciling of viewpoints.
When I first hit the quota limits, I tried to setup caching. Failing that, I tried setting up a second LFS server and having CI pull blobs from that first when pulling simple incremental commits not touching said blobs. Details escape me this long after the fact - I might've tried to redirect LFS queries to gitlab? After a couple hours of failing to get anywhere with either despite combing through the docs and trying things that looked like they should've worked, then I tried to pay github more money - on top of my existing monthly subscription - as an ugly business-level kludge to solve a technical issue of using more bandwidth than should really have been necessary. When that too failed... now you want to pin the whole problem on github? I must disagree. We can't pin it on the CI provider either - I had trouble convincing git to use an alternative LFS server for globs when fetching upstream, even when testing locally.
I've tried gitlab. I've got a bitbucket account and plenty of tales of people trying to scale git on that. I've even got some Microsoft hosted git repositories somewhere. None of them magically scale well. In fact, so far in my experience, github has scaled the least poorly.
> Github the company is not interested in providing you (or anyone else) with free storage for arbitrary data.
I pay github, and tried to pay github more, and still had trouble. Dispense with this "free storage" strawman.
> You were unable to pay for the storage options they do provide because you did not have admin rights to the github account you wanted to work with.
To be clear - I was also unable to pay to increase LFS storage on my fork, because they still counted against the original repository. Is this specific workaround for a workaround for a workaround failing, github's fault? Yes. When git and git lfs both failed to solve the problem, github also failed to solve the problem. Don't overgeneralize the one ancedote of a failed github-specific solution, from a whole list of git problems, to being the whole problem and answer and it all being github's fault.
> None of this is a problem with git, be it GUI git clients or command line ones.
My git gui complaints are a separate issue, which I apparently shouldn't merely summarize for this discussion.
Clone https://github.com/rust-lang/rust and run your git GUI client of choice on it. git and gitk (ugly, buggy, and featureless though it may be) handle it OK. Source Tree hangs/pauses frequently enough I...
I'm conflating at most one github specific issue (singular), not "issues". And I'm doing so because it's at best a subproblem of a subproblem of a subproblem.
If my computer caught fire and exploded due to poor electrical design, you wouldn't say "nothing about your problems had anything to do with your computer and everything to do with the specific company that provided your pencils" when in my growing list of fustrations I offhandedly mentioned breaking a pencil tip after resorting to that, what with the whole computer being unavailable and all. That would be weird.
Even if we did hyper focus on that pencil - pretty much every pencil manufacturer is giving me roughly the same product, and the fundamental problem of "pencils break if you grip them too hard" isn't company specific. It's more of a general problem with pencils.
Github gave me a hard quota error. Maybe Gitlab would just 500 on me, or soft throttle me to heck to the point where CI times out. Maybe Bitbucket's anti-abuse measures would have taken action and I'd have been required to contact customer support to explain and apologize to get unbanned. git lfs's fundamental problem of being difficult to configure to scale via caching or distribute via mirroring isn't company specific. It's more of a general problem with git lfs. Caching and mirroring are strategies nearly as old as the internet for distribution - git lfs should be better about using them.
It would've turned github's hard quota error into a non-event, non-issue, non-problem - just like they are with core git. Alternatively, core git should be better about scaling. Or, as a distant third alternative, I could suggest a business solution to a technical problem - GitHub should be better about letting me pay them to waste their bandwidth. Then I could workaround git's poor scaling for a little bit more, for a bit longer.
A major limitation of git is how it deals with many "big" (~10Mb) binary files (3D models, textures, sounds, etc.).
We ended up developing our own layer over git, and we're very happy ; even git-lfs can't provide similar benefits. This technique seems to be commonplace for game studios (e.g Naughty Dog, Bungee), so certainly git has room for improvement here.
Has any studio open sourced what they've built? Or turned it in to a product? It seems there could be an opportunity to do something before git solves the problems for that use case.
Jup, I experienced this too (not at a game studio though, and the team I worked with wasn't nearly experienced enough to write a layer over git). When we switched to a new version of the git gui it would stop working because when you click through the GUI, it would perform some git operations that were supposed to run fast. I filed an issue that quickly got shot down with 'wontfix, your repo is too large and git is not for binary files'.
Do you have tools that you can utilize diffs from your binary file changes? Or does a change simply just replace all the bytes.
I'd argue if it's the later, that git was never the right choice to begin with. You don't really want to record a full 10MB of data every time you change one pixel in your texture or one blip in your sound, right?
So I don't know if this is a "major limitation" of git per se. Not saying there's a better solution off-the-shelf (you're obviously happy with your home grown). But this was probably never a realistic use for git in the first place.
While I can't speak for the person you're replying to, the technology at least exists. Binary diffs are sometimes used to distribute game updates, where you're saving on bandwidth for thousands if not millions of players - which costs enough $$$ to actually be worth optimizing for. On the other hand, between simpler designs and content encryption being sometimes at odds with content compression... so is just sending the full 10MB. For a VCS - I'd probably be happy enough to just have storage compression - using any of the standard tools on the combination.
> You don't really want to record a full 10MB of data every time you change one pixel in your texture or one blip in your sound, right?
Actual changes to content in a gamedev studio are very unlikely to be as small as a single pixel. Changes to source code are unlikely to be as small as a single character either. And we definitely want a record of that 10MB.
We're willing to sacrifice some of our CI build history. Maybe only keeping ~weekly archives, or milestone/QAed builds after awhile, of dozens or hundreds of GB - and maybe eventually getting rid of some of the really old ones eventually. Having an exact binary copy of a build a bug was reported against can be incredibly useful.
> Sure, immutable build artifacts can be invaluable -- but aren't they also an orthogonal concern?
One person's immutable build artifact is another person's vendored build input.
It's common to vendor third party libraries by uploading their immutable build artifacts (.dll, .so, .a, .lib, etc.) into your VCS, handling distribution, and keeping track of which versions were used for any given build. It makes a lot of sense if those third party libraries are slow to build, rarely modified, and/or closed source - no sense wasting dev time forcing them to rebuild it all from scratch.
The next logical step is to have a build server auto-upload said immutable build artifacts into your VCS, for those third party libraries that you do have source code for, when your VCS copy of said source is modified. Much more secure and reproducable than having random devs do it.
And hey, if your build servers are already uploading build artifacts to VCS for third party libraries, why not do so for your own first party build artifacts too? Tools devs spending most of their time in C# probably don't need to spend hours rebuilding the accompanying C++ engine it interoperates with from scratch, for example, so why not "vendor" the engine to improve their iteration times?
This can lead to dozens of gigs of mostly identical immutable build artifacts reuploaded into your VCS several times per day, with QA testing and then integrating those build artifacts into other branches on top of that. The occasional 10MB png is no longer noticable by comparison.
To be fair, I mostly don't fault git for failing to optimize that far, even if there are alternatives that do. That's far enough outside the core use case for myself and those I know that I'd be willing to sacrifice it for other, more important considerations.
But I'm totally willing to fault git for failing to optimize enough to handle the manual commit cadence of source game assets though. Because that's not just a tertiary use case - frequently for coworkers it's their primary use case. The end result is I mostly only use git for personal hobby stuff, where it's a secondary use case and my assets are sufficiently small as to not cause problems.
I can sympathize with the game assets argument, but this problem is just the result of trying to stuff a square peg into the round hole.
Build artifact caching is a different problem from source control, with very different requirements:
1. As you mentioned, the artifacts tend to get huge.
2. The cache needs to be easy to bypass. From your example, it needs to be easy for the C++ engine devs to do builds like "the game but with the new engine" to test out their changes.
3. The cache needs to be precise, so you don't end up with mystery errors once it finally does trigger, or people wondering why their changes don't seem to apply.
4. The builds need to be exactly reproducible, so you don't end up with some critical package that only Steve Who Left 5 Years Ago could build (or Jenkins Node 3 That Just Suffered A Critical HDD Failure).
Git either doesn't care about or fails spectacularly for each of those points. In particular, #3 will be very confusing since there will be a delay between the code push and the related build push.
Nix[0] solves #2 and #3 by caching build artifacts (both locally and remotely[1][2][3]) based on code hashes and a dependency DAG (for each subproject or build artifact, so changing subproject X won't trigger a rebuild of unrelated subproject Y, but will rebuild Z that depends on X). It helps with #4 by performing all builds in an isolated sandbox.
#1 is solved by evicting old artifacts, which is safe as long as you trust #4. If the old artifact is needed again then it will be rebuilt for you transparently. Currently this is done by evicting the oldest artifacts first, but it could be an interesting project to add a cost/benefit bias here (how long did it take to build this artifact, vs the amount of space it consumes?).
Assets and code have mostly the same needs out of a version control system - diffs, history, control over versions, etc. - and there are version control systems which handle both adequately. That said, I'll grant git is quite focused on code version control specifically - and I would not dream of trying to scale assets into it directly.
> 1. As you mentioned, the artifacts tend to get huge.
This, admittedly, is more common with build artifacts. That said, I've hit quota limits with autogenerated binding code on crates.io, with several hundred megs of code still being in the double digits when better compressed by cargo than I can figure out how to compress with 7-zip.
And that's a small single person hobby project, not a google monorepository.
> 2. The cache needs to be easy to bypass
I need to bypass locally vendored source code frequently as well, to test upstream patches etc.
> 3. The cache needs to be precise, so you don't end up with mystery errors once it finally does trigger, or people wondering why their changes don't seem to apply.
Also entirely true of source code.
> 4. The builds need to be exactly reproducible, so you don't end up with some critical package that only Steve Who Left 5 Years Ago could build (or Jenkins Node 3 That Just Suffered A Critical HDD Failure).
Enshrining built libs in VCS is an alternative tackling of the problem. You might not be able to reproduce that exact build bit-for-bit thanks to who knows what minor compiler updates have been forced upon you, but at least you'll have the immutable original to reproduce bugs against.
> In particular, #3 will be very confusing since there will be a delay between the code push and the related build push.
It's already extremely common - in the name of build stability, including with git - to protect a branch from direct push, and have CI generate and delay committing a merge until it's verified the build goes green. By wonderful coincidence, this is also well after CI has finished building those artifacts - in fact, it's been running tests against those artifacts - so it can atomically commit the source merge + binaries of said source merge all at once. No delay between the two.
There are some caveats - gathering the binaries can be a pain for some CI systems, or perhaps your build farm is underfunded and can only reasonably build a subset of your build matrix before merging. Or perhaps the person setting it up didn't think it through and has set things up such that code reaches a branch that uses VCS libs before the built libs reach the same spot in VCS - I'll admit I've experienced that, and it's horrible.
Nix, Incredibuild, etc. are wonderful alternatives to tackle the problem from a different angle though.
I kind of phrased that poorly. I should have added the context of "in git". Saving a new 10MB file every time you change it, as per my original premise, is not something that git was really designed for. It's asking a screwdriver to do the work of a hammer.
I totally get the use case of saving each iteration of that 10MB file _somewhere_. But expecting git to do that job is not the right level of expectation, was my main point.
When I have worked with binaries like that described, I will place a URI reference to a file hash and have something that knows how to resolve it. A file store (think S3 or whatever) that has files named: texture1.dat-[sha1] and change the reference to the file in the source. e.g. a "poor man's" version control by way of file naming conventions. Does this approach work in your world?
This does not surprise me. Git's original purpose of managing versions of a tree of text files (i.e. the source code of the Linux kernel) pervasively influences it, and I wouldn't expect it to be any good for working with binary files or large files.
If somebody comes up with something that matches Git's strengths and also handles binaries and biggies much, much better then they could definitely topple Git with it. It'd take time for the word to spread, the tools to mature and the hosting to appear, but I can definitely see it happening.
I think most people know that Git isn't perfect, but it's also the case that coming up with anything better is an extremely difficult task. If it wasn't, someone would have already done it. It's not like people haven't been trying.
Aren’t game studios and other creative studios meant to use “asset management” systems instead for their large binaries?
Diffing a PSD as a binary is impossible - whereas proper asset management tools will deconstruct the PSD’s format to make for a human-readable diff (e.g. added/removed layers, properties, etc).
Separate version control for code vs assets leads to a world of pain. Also you can use whatever diff tool you want; doesn't have to be the built-in textual diff.
> I disagree that Git's biggest flaw is its lack of scalability. Cases where git needs to scale tend to be isolated to companies that have the manpower to build a finely-tuned replacement (see: MS, Google).
I would say that the sole thing git was developed for, the Linux Kernel, is (starting to be) painful to work with when using git.
The Linux Kernel is big, but it's not likely as big (in terms of lines of code or pick your metric) as Google or Microsoft repositories. Maybe the kernel is just starting to feel that pain?
Honestly asking.. Do you speak from some level of authority that the Linux kernel is stretching the boundaries of git? Or are you just saying that more speculatively? What is the painful part?
Maybe I am a weirdo but I have always thought that git's UI is very intuitive (with some exceptions like sub modules). SVN on the other hand was an unintutive mess where I had to look up commands all the time.
Agreed - I had been looking around this thread like, "<slow blink> - surely I'm not the only one that finds git to be a rewarding exercise in teamwork?"
The magic sweet spot might be the fact that most projects to not need to be distributed. This is where a lot of complexity is derived.
So without all those extra concerns - and - a more elegant UI framework (i.e. rational commands) - and possibly something that scales a little better. That's enough mojo to unseat git for a lot of things.
> Cases where git needs to scale tend to be isolated to companies that have the manpower to build a finely-tuned replacement (see: MS, Google).
I recollect that for Windows (which also uses git), MS have actually extended git with "Git Virtual File System" rather than replace it[1]. But I do agree that broadly, not everyone needs the scale.
What are you smoking? GitHub and gitlab together basically encompass almost ALL of humanities efforts with regards to software development. Doesn't scale??? You are seriously delusional.
"Barney Oliver was a good man. He wrote a letter one time to the IEEE. At that time the official shelf space at Bell Labs was so much and the height of the IEEE Proceedings at that time was larger; and since you couldn't change the size of the official shelf space he wrote this letter to the IEEE Publication person saying, since so many IEEE members were at Bell Labs and since the official space was so high the journal size should be changed."
What is the analogy here? The first guess that came to my mind was the monorepo versus multi-repo debate: since Git can only support repos that are so large (shelf space) without getting slow, you should split up your repos (journal size) even if semantically you would prefer a monorepo. But that would support the point I was making, whereas the obliqueness of your reply makes me think you probably meant to criticize it.
I think he's comparing the journal to the tool (harder to change, impacts everyone) and the shelf to the problem that only impacts a few organizations but actually a lot of people because those organizations are so large.
I guess that makes sense. But if that's the analogy, there's a significant difference between the situations. In that example, there was nothing inherently wrong with the journal's size, other than it not matching Bell Labs' arbitrary choice of shelf layout. Git, on the other hand, would be inherently a better tool if it had better performance on large repos (without sacrificing its suitability for small repos).
Mercurial is probably that competitor. Only slightly slower than Git. Works on very large monorepos (as large as Facebook's or Google's monorepo). Very similar workflow as compared to Git, with some minor differences in terminology.
As an FB employee, I use hg regularly (because it is required). I would not use it as a git replacement for non-FB-sized repos. It has some weird design choices (e.g. branching is bad), and it very often requires manual intervention for merges that git performs correctly and automatically.
You can get around branching-is-bad by changing your workflows a bit, but you can't get around the bad merges: over time it's like death by a thousand papercuts.
What kind of merges does git handle that hg doesn't? If it's just a matter of figuring what goes where, someone that uses hg daily could copy the implementation from hg. It could be a big organization that uses daily for instance.
Paper cuts can be addressed with more users reporting bugs and contributing fixes. The fundamental design issues with git that prevent scalability cannot.
As a Google employee I use hg every day, even though it's not required. (Some teams at Google do mandate its use, but these are few and far between.) I don't use branches, but I use bookmarks. I didn't notice any merges that really ought to be performed automatically but were not; in any case I use Meld to resolve merge conflicts and it's easy enough to do occasionally.
For most people, avoiding thousands of papercuts is better than scaling massively. Few people need massive scale, but everyone hates papercuts.
I'm also not certain that the "fundamental design issues" with git are truly fundamental to its design. For example, partial clones and sparse checkouts are seeing increasing support in recent versions of git — and those are really all you need.
You can always strip a bad committed merge, abort a bad uncommitted one, and perform it again (maybe with different tooling).
Normally mercurial stops when there are conflicts it cannot resolve reliably. In that cases, have a try at kdiff3: it handles hairy merges quite well. In a lot of cases even automatically (and correctly).
There is always meld, but I'd say kdiff3 is superior wrt merge conflict resolution.
What is so bad about mercurial branching? The underlying structure is the same as git: a directed acyclic graph, the only real difference is how branches are named.
Mercurial has 3 ways of doing branching:
- bookmarks: these are like git branches, a pointer to a revision
- branches: when you are in a branch, all commits are permanently affixed with that branch name. Less flexible than bookmarks (and therefore git branches) but good for traceability
- heads: unlike with git, a branch name can refer to several actual branches, it usually happens when you are pulling from a central repository, but you can create them yourself if you need some kind of anonymous branching. These can be pushed but it is not recommended.
Git only has the first option.
The way central repository are managed is also a bit different even if the fundamentals are the same. Git has the "origin" namespace to distinguish remote branches from local branches. Mercurial uses a "phase" which can be "public" (in remote), "private" (local only, will become "public" after a push) and "secret" (like "private", but will not be pushed and therefore will not become "public"). So if you are not synchronized with the remote, in git you will have two branches: origin/my_branch and my_branch, in mercurial, you will have two branches named my_branch, one public, one private. That's essentially the same thing, presented differently.
In the end, they are fundamentally the same. The feel is different though. Git is flexible, and gives you plenty of tools to keep things nice and clean when working with large, distributed project. As expected for something designed for the Linux kernel. Mercurial focuses on preserving history, including the history of your mistakes, and I feel it is better suited for managed teams than a loosely connected community.
Specifically I meant that branching is less flexible. Bookmarks are better than Mercurial branches (and at FB it's what we use instead), but even with bookmarks there are gotchas compared to git. For example:
* Pushing them is (slightly) more annoying than pushing git branches — you need a separate command, whereas `git push` just does the right thing by default
* Deleting them doesn't delete the corresponding commits
* There is only one global namespace shared across all remotes
That's interesting. In your examples isn't it fast because monorepos are network-based, as in, you only fetch what you need when you need it?
Also reminded me of discussions around CPython's startup time and how one use case where milliseconds matter is in small cli utilities such as Mercurial.
What I mean is daily operations on the repo like viewing a diff, committing, amending, checking out a different commit, etc. Without doing precise measurements, I would tend to think that it's mostly caused by the slowness of CPython, as compared to a C executable (Git).
The entire repo is stored on a networked file system. So essentially every file operation is remote. That doesn't actually contribute to much slowness because when I didn't use hg, operations were noticeably faster.
The only way I've ever successfully `git clone`d my work repo is from another locally connected device. Even with shallow and then gradually unshallowing it, it will not generally complete before the internet falls over.
Nowadays, a new computer means a git clone (or just plain copy-paste) of a USB stick from the old one. This seems like it's a single feature of git that could be written, but if you told me "there's something that works better for large, twenty year old repos", I'd probably take that.
I don't know how Linux survives, but maybe it's just that you only rarely git clone your large repos. (Or maybe it's just that intercontinental internet is less reliable than intracontinental, so that if you're in the US it's a non issue.)
Git scales well enough for almost everyone (especially if you have a little discipline with what you put in the repo).
It’s only huge megacorps that need larger scale things like GVFS.
As for large files, that is not what Git is for. Git is for source code. Much like how you don’t put large files in your RDBMS, you should not be putting them in your SCM either.
What if you need to version them? Git imposes a very specific versioning model: version is a property of the entire repository. Thus, not including some file in the repo implies that it's not versioned in the same manner. It's not just a function of binary vs source.
Versioning big binary blobs is not what Git was designed for. It’ll do fine with smaller assets like icons and the like, but its data model is based on everyone using the repo having a local copy of the full repository history. You can’t easily purge old data. That scales poorly if you want to use it for audio/video files or other large data sets.
You can still do it if you want, but you might be better served using https://git-lfs.github.com/ or using another system designed for that purpose.
Honestly, you can use Git for large files with lfs. I wouldn't say I love this approach, but it isn't that bad now. You do have to make room for yet-another-tool, and you now have centralized version control comingling with your distributed tool (essentially making it central); but you can still use everything you love about git, and if your lfs doesn't change, you don't need to be connected to a server. It certainly feels pretty absurd. This isn't even a problem in SVN, but now we're tacking on another tool that you have to learn, and introduces issues.
I believe there will be no scalable open-source VCS because the incentives are not there. While the technical problem is interesting, I decided not to work on it because of this. http://beza1e1.tuxen.de/monorepo_vcs.html
When people talk about killer features missing in Git, there is more beyond the UX and mono/poly repo.
One thing is code review. There is no code review in Git.
What I expect in 2020 is that I should be able to specify reviewers for the commit (which I pick out of a list of people who can approve it). These people should be able to leave comments on the commit. I should be able to both respond to comments and modify the code before the commit gets checked in. The history of the comments and changes should be maintained.
There is nothing in Git that supports this flow in a natural way.
A replacement for (or evolution of) Git can be a tool that would support this code review flow from the get-go.
And yes, there are external tools for code review. But that all should be a part of version control.
I think it doesn't exist because there is no demand for that to be part of the codes history. Nothing you've described sounds very useful after a month or so.
That's the one major point where Fossil probably falls short: Extending a repository's capabilities would add overhead to every single one of them (and, potentially, to each commit's meta data).
All this talk of Git-sympathetic code review tools and nobody has mentioned Gerrit, which seems at least somewhat close to what is being described. Each patchset of each review is its own Git ref and is often referenced in the final commit.
Separately, I know many including myself who would love for code review comments to more seamlessly be integrated into the code browsing experience.
People also seem to be conflating review comments associated with lines of code with an opinionated code review tool.
You can keep the banter with the code and the go-no-go decision separate, even external. But post mootems have worked better when someone realized that one of the team repeatedly calls out a class of errors that bite us later and they’re being ignored. You have the ability to prevent this error. Wise up or that person will decide rightfully that we are a bunch of clowns and leave.
Code review comments really should live next to the code, and proper editor integration goes a really long way. Traditional review processes make “just fixing the code” have a very high inertia.
I don't think this should be part of version control, simply because is too tied to the environment and development practices that may not be shared by the whole set of current and future developers of any given project.
The version control should keep the code history, not the paperwork history.
What I think you're looking for could, however, use git as a platform for that. That's what GitHub, GitLab and the likes do using a web interface and there's enough extensibility and power on git for command-line or desktop tools to do it.
The internals of git are very much akin to a filesystem, by the way, and the plumbing gives you more enough access to use that in creative ways.
From the top of my head, maybe a system like this could automatically generate tags for code reviews, use merges and branching for answering to these reviews, and hooks for notifying interested parts on particular areas of code. All while the messages and reviews themselves travel on another data layer, which references git but does not mingle with it.
This separation (and even tag cleaning, for example), would be specially useful on huge distributed projects where a company or small team may have whatever development process it needs internally and sharing only the results without having to completely hide the code history.
I think the distinction you're drawing between code history and paperwork history is more arbitrary than you give it credit.
If all we cared about was code history, a super pure "version control" system would have one trunk, no branches, and a sequentially increasing version number with no commit messages or author information.
But if you can annotate an entire commit with a descriptive message, why not annotate a specific hunk of code changes (comments), or the discussion over those changes in a specific version?
Environments and development practices may change, but the code review that led to a particular code decision at some point in history are a valid representation of the context at that time in history.
Don't get me wrong, I'm not lacking these features in Git, and am happy to get them from other platforms like Github, but I think the comment you're responding to is astute that one can imagine a git replacement that incorporates code review functionality as a first-tier feature.
Perhaps commits should support key:value metadata? (I was about to say "tags", but that means something different here) That would let you support "reviewer:person@foo.bar" or whatever you want, without baking workflow assumptions into the VCS.
> I think the distinction you're drawing between code history and paperwork history is more arbitrary than you give it credit.
I don't agree. The responsibility of a version control system is to manage the changes made to the source code. The paperwork that goes with it is an entirely different, and separate, responsibility, just like ticketing systems and keeping track of tasks, epics, sprints, etc.
> If all we cared about was code history, a super pure "version control" system would have one trunk, no branches, and a sequentially increasing version number with no commit messages or author information.
That doesn't make sense because you're ignoring the fact that branches are used to host versions that are being developed independently at a specific moment in time, and merges are used to finally join contributions when they are ready to be added to the main branch. If you look at single branches and ignore the work being developed in any other branch, the history is as linear as you expected it to be.
> But if you can annotate an entire commit with a descriptive message
If you can annotate an entire commit with a message, why not allow annotating parts of a commit (file/span) with a message, and why not allow those messages to form threads?
At which point you get code review that is logged entirely in the history of the repo. Which is a very useful feature, and a huge value proposition of GitHub over raw git - git blame gives you the commit, but GitHub will also gives you the pull request for that commit, and you can go and look at the comments there to understand why it was done the way it was.
You could actually do that today with the Git model if you want to, so, arguably, the question is, why doesn't your code review tool already put its history in git?
(Probably some combination of nobody asking for it, and in some cases, wanting to keep your code review process locked in to the tool.)
I kind of agree with grandparent, though. When the IDE, source control, code review and task tracking work together seamlessly, it's glorious. If your software says "it's not my job", it will be outcompeted by software that says "heck yeah it's my job".
I would like a workflow with all of the features of GitHub (code reviews, issue tracking, etc), but everything stored in Git.
Perhaps every project might have a branch called "issues" or "reviews", etc. Not sure the best setup but generally the more I can do in my code editors the faster I can work.
I disagree. Git handles this through the power to push/pull and muddying that with a bunch of alternative baked in user flows seems like a mistake. Third party software can handle adding meta-information to pull requests.
> And yes, there are external tools for code review. But that all should be a part of version control.
There are many different code-review workflows, and for teams that do full-time pair-programming, code review happens in real-time as the code is written.
Trying to bake support for all of that into something that is also a good VCS sounds like a recipe for one of those clock-birdfeeder-machete-flashlight-massager tools that you used to see on the back pages of outdoor magazines in the 90s.
This is exactly the issue with it: its author(s) lament that git has not all of these things integrated, but they are ignoring that this is exactly why fossil cannot gain a wider adoption.
It has opinions about how the team and the project should be run / managed / documented. This is not for the tool dev to have these opinions
Yes there is. You can block pushing directly to a branch of the repository and demand that things can only be merged through pull requests. You code review the pull requests.
This is very well supported by Atlassian's git tooling, for example. It does everything you mention. It's true that the comments don't become part of the git repository itself, but I'm not convinced they should be.
Eventually git will be replaced by something better. It might not be replaced everywhere, but eventually it will be replaced in most places. This is true of almost all things in tech.
I expect that git's eventual replacement will initially boast compatibility with existing git repos.
If you take a look at Subversion vs. Git's interest on Google [1] and you agree it has some correspondence to the technology adoption lifecycle [2] ... Git's got a long way to go.
I wish there was a version control system that didn't need user input beyond a "push". Only branches and pull requests, nothing more. No commit messages, you have one master branch and however many sub-branches you need.
Whatever code is in your branch is what it is. And then there's a layer of magic on top of it all, maybe a UI or command line tool or both, just to be able to easily rewind time. Either per file, per line of code, or folder, or just a folder but not recursive, or the entire branch.
Think of Apple's Time Machine, it should be that simple.
Honestly, I rarely–if ever–read commit messages to begin with. The way I navigate older code is never done by searching for commit messages. That's not reliable.
Instead I simply go to a point in time where I think the thing I'm looking for might be. I'll look at the code, recognise its state, and continue the search up or down the timeline.
And that would cover the needs of most projects I'd say. And it would save us a shitload of time.
Hell, I've been working with git for almost a decade now. I never needed to rebase or merge things, until I recently did need to do it. It's too arcane to make intuitive sense, to me anyway.
Git is good, I'd just welcome a breath of fresh air...
Do you work on a team? Or in a job were you need to have an audit trail for your work? Commit messaged and many other features of Git like branching and tags are indespensible for sharing and collaborating on code.
Flip your questions around. Might someone who isn't on a team, and doesn't need an audit trail, want something simpler?
Versioned files systems, including Apple's Time Machine and Dropbox's version history, have no extra UI to save versions. A small, short-lived project with at most a few collaborators (eg, working on a small scientific paper) might find those more useful than git or other VCS.
I would have to agree with this. I have been programming for decades and I haven't had to work on a team for a project. "Branches" have not happened, nor have I ever felt the lack of them.
I would not mind something simple. I find the arguments, flags, whatever in git to be rather opaque. That, coupled with the lack of need for its features, have not thrilled me.
It seems to be pretty effective if you have tons of programmers working on a single project, but at the bottom end of the scale, I find it baffling.
I particularly enjoy IntelliJ's local history, which will go so far as to tell me whether the tests were passing or failing (and how many failed) on a specific file at a specific point in time.
Bit Git isn't great for audit trails given that you can go back and alter previous commits, in contrast to pretty much any other VCS, including Subversion and Fossil.
(I know, technically, altering commits creates new commits in Git, but it comes to the same thing.)
Taking the question literally, obviously not. If git became a problem due to some unforseen licensing issue or whatever mercurial does the job just fine right now and has for years.
As a heads up if you're stuck on cvs, svn or other stupid vcs due to "old-codgers" in your office, mercurial has a shallower learing curve and easier ui to get the same job done which may make it easier to switch.
Either git or mercurial, who cares? Either of them until there's something better. Never deal with SVN & CVS branches and merges again. Feel the immediate team productivity boost which will pay for the initial learning curve costs by day 2. Seriously.
Unfortunately, you have to settle on some workflow for that. And this settling will take some experience and discussion. So "day 2" sounds very overambitious to me.
Use the existing svn/cvs workflow with the trivial changes for git/mercurial. Central repo, push when you're done to master. Branches are all but unusable on cvs/svn anyway - if you do use them, feel the win and laugh with glee. When everyone is on top of that basic workflo then you do something incrementally better. Really it's day 2, really. I've seen this more than once now. The fight to get to that point where the changeover happens is getting easier but I'm sure it still sucks.
One major fundamental issue with git that nobody has brought up is that its data model is close-to-incompatible with some of the current legal and moral requirements around data privacy today. For example, as far as I can tell, GDPR allows any European citizen who has ever committed to the Linux kernel to request that their name be permanently expunged from their contributions, and everyone with a git clone of the kernel is legally obligated to perform a rebase to do so. It's possible to make a DVCS where that is an easy operation, but git very much isn't it. That means it can't be used to store personal data, and even standard corporate policies of "we delete old historical records so they don't bite us in court" aren't supported.
There are definitely times you want to store the full history forever, but it would be nice to have a DVCS that gave other options.
The legal requirements in the EU are at odds with the moral requirement. Changing or erasing history in Git is an anti-feature. Checked in secrets should be assumed to be compromised and changed immediately. Removing contributions should be treated as any other form of censorship.
Those who forget (or rebase away) history are doomed to repeat it.
Does Fossil do sub-modules correctly? Specifically does it let you compose a project of several repositories and do configuration management on that repository? That is something git kinda sorta does but it gets out of control easily.
I like the idea of storing commit history in a database. Why write all of the bits from scratch?
But I don’t like how difficult databases make it to represent graphs of data. To pull a subtree cheaply, you need a graph. not unlike SVN’s data structure.
It is something that I prefer to have in a VCS. At one end of the spectrum you treat a repository as a object which a containing repository can subclass with specific changes. Such a system would allow things like basic driver frameworks for a bunch of devices to be in their own repository and the 'critical bits' that make them work in a system of type 'x' would end up being a that repository + changes. You can kind of do this with git where you create a branch for each system x, y, and z in the repository and then your submodule is driver repo branch x but that doesn't put the maintenance burden in the right place (the system X maintainer).
Fossil's intended use case is different from Git's; it was originally built as the VCS for SQLite. In terms of its target audience, it's basically GitHub-in-a-box for small or medium-sized teams. Implementation/usability-wise, it does some things better than Git, does other things worse, and yet others simply different (in a way that some may find better, some may find worse).
For what it's worth, the linked page is itself running on Fossil, which has of course build in forum facilities and several other suchlike goodies - being a complete solution, packed into a single smallish executable with no significant dependencies apart from SQLite. The Fossil site runs on very moderate hardware, and gets no hiccups from hitting the HN frontpage.
I like this kind of untroubled minimalism, and so far have never encountered a reason not to run every personal project on Fossil. The real world will occasionally force me onto Git territory, but I can't really say I have ever enjoyed the experience.
Git is here to stay for any foreseeable future, of course. And while I do understand points often made about the benefit of one de facto standard to rule them all, monolithic dominance always tends to unsettle me. My SE friends in general simply use 'Git' as a given synonym for 'version control'. And it does annoy and somewhat worry me that they've never even heard of Fossil until I roll out my sermon.
We use Fossil as well @work and we love it, although I don't think it's not the best choice for huge codebases such as the FreeBSD ports tree which I tried to import once to see how it scales and gave up after >2Gb and a hour or crunching. Maybe importing into nested repositories would work better?
558 comments
[ 2.6 ms ] story [ 399 ms ] threadThe article is basically arguing that we're likely to see the same state of affairs in VCS land. Sure, there will still be new ones coming into use, and fading out, or finding small niches - but Git will remain dominant for decades to come.
IMHO, the ability to “commit early, commit often” and to squash/rebase original work later into clean and understandable commits is one of the best features of Git.
The version control history is made for reading by other people, so it is a story. After all, we only write the commit once, but people will read it many more times. (This is especially true if there are code reviews involved)
I am an imperfect programmer. My work-in-progres is often broken, and even fails to compile. Sometimes I will refactor interface only, and will want to checkpoint my work before I go and refactor implementation as well. Sometimes I will choose a totally wrong approach and revert it later. Sometimes I will disable/break large part of system on purpose, to make testing easier. And I often do stupid data-destroying mistakes, so I want an ability to save/store all the past versions, even if they are completely broken.
My “raw” commits may look like: “start on feature X”, “refactor interface Y”, “more work on feature X”, “wip commit”, “fix tests”, “fix performance”, “fix more tests”. Does any future reader care it took me 3 commits to get the tests right? Do they care that I discovered the need to refactor only while I was halfway in feature X implementation? Do they want to see a repo that won’t even compile? Do they want to hav to cherry-pick dozens of commits to get the tests to pass? I don’t think so.
The final version will only have two commits, “refactor” and “feature X”. It would be obvious to everyone which lines of code are associated with which change. Each revision will be buildable, and will pass all tests - so bisect will actually work.
(If rebase support is missing, it is possible to “fake” it by having multiple checkouts and manually copying files around. But this is much more error prone and dangerous. I have spent plenty of time with SVN/CVS, manually copying files and applying patches - and I can tell that having this integrated with version control is much more pleasant)
It does huge well, and I like the one-number changelist.
This is also very useful for working with external contractors. Not only can a user specify their own view specs, there is admin control for client views that can make portions of the repo read-only or effectively invisible on a per-user basis.
Perhaps this is possible with git?
[0] http://www.catb.org/~esr/writings/world-domination/world-dom...
[1] http://xent.com/pipermail/fork/2002-January/008429.html
Git is somewhat confusing to use, but not enough so that anyone really cares all that much (besides a few people who really care) and that is not a recipe for easy replacement.
There were/are slightly less confusing version control systems (mercurial) but they didn't catch on for whatever reason.
I would say the other kind of for loop, a “for x in y { body }” loop, is very intuitive and obvious to most English speakers. We use idioms like “for each A, do B” all the time.
Also (sorta) darcs, and hopefully pijul. I sincerely hope some patch-based version control can catch on.
pijul regrettably seems to be broken most of the time, but I don't think it's irredeemable.
The general consensus is that putting big files in git means you're doing something wrong and the problem is with your environment not git. (or there are special-purpose tools for your kind of workflow which handle the specifics of your use case, like CAD/CAM/etc.)
Workflows like that generally don't fit into nice little boxes anyway the way source code management does.
This is not a “general” consensus. It’s a consensus among hardcore proponents of git. I love git, it has made my life better. I still think it’s large file support story is shitty/suboptimal, and there are valid use cases where a general purpose VCS is used to track large binary assets along code and git would do well to be a general purpose VCS. It’s a limitation of git. It’s not a fatal limitation, and git still has enough benefits (which include availability and mindshare), but it is still an unfortunate limitation and somewhat ironic for a tool born in a world where everything is just a “sequence of bytes”.
I don't see files going anywhere. I would like to listen to examples of "code-in-database is already halfway here", if you happen to have them.
Low-code is very suddenly going to be a big thing in the next couple years and when it happens there are now 750M more "coders" and those types of people are not interested in git pull --rebase --fucked --whatdidido
File systems haven't been disappearing, they've just been getting hidden from users. I have seen exceedingly little evidence that anything beyond merely hiding them is actually happening in the industry. "Stick all the files in databases" seems to be seeing no penetration beyond music collections, which at this point, I think we can call a "mere use case" rather than the vanguard of a revolution, since it's a good 10-15 years old minimum.
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=106080
That problem I saw even before I saw a replacement.
But once I saw git, having a full copy of the repo was pretty killer. It was also better at merging, partially for having the history local. In fact I recall some SVN people protesting at the time that merging wasn't so hard, but in a nutshell, they were just wrong. It was harder. Source control that can't merge fluidly is pretty limited from the get-go. There's literally entire dimensions of things we do with git that were so impractical with SVN that only the very largest projects could afford to do them, with the whole "workflow" question. Now you don't even hear about the difficulties of merging since all the open source SCMs copied each other and "good merging" is just table stakes now.
The problem is is that git doesn't really have fatal flaws, it has annoyances, and that's not the same thing. "The UI is difficult" is an annoyance for its target audience. Most every other criticism I've seen of it is an annoyance, not a fatal flaw. Fossil doesn't solve a problem I have with git. It has some neat ideas and arguably does solve some problems, but it doesn't solve the problems I have with git; you could lift Fossil's solutions to wiki and bugtracking and just put them in Git and I'd probably be happier with that than Fossil itself.
The problem isn't that Fossil is an SCM competing with git; the problem is that Fossil is a monolith competing with the rapidly-moving git ecosystem. The former is perhaps beatable, the latter is a juggernaut. All of Fossil's other features sitting in git is something that might get somewhere, but if the base source control isn't git-based, I can't sneak it into a work project to try it out. I've got a corporate mandate that all source belongs somewhere standard, and it's a perfectly sensible requirement for someone paying me to do a job.
that is a feature! You don't know how I miss that!
(that, and the handling of blobs)
Handling of blobs I can at least see preferring the differences, though git-lfs mostly acceptably shims that for me. The need to use git-lfs is definitely an annoyance, but it's another place where it's probably not a fatal flaw.
Or, to the extent that it arguably is, it would be when you're using git as something that it really, really isn't... it's a source code management tool. It's a good enough source code management tool that it's useful for many other things as well, like the way some people directly drive their personal websites with it. But it isn't a tool for managing lots of large files that can't be textually diffed, not a generic "content" management tool. I wouldn't expect git to necessarily store terabytes of video files, but then, it's not like Fossil is going to do that either from the sounds of it. git-lfs shims content blobs well enough to make it just an annoyance for source code management.
Yep, handiest way to perform a first pass on narrowing down an introduced bug between a few commits.
Fossil has bisecting with much the same CLI as Git, for what that's worth.
Sure, not being able to rename a file easily in CVS was quite an annoyance but nothing that made it completely useless.
There were always workarounds. But exactly those areas were workarounds or better tooling patches over stuff Git doesn't do are the areas that a possible successor will address.
That's just quibbling about definitions in fuzzy English. If you want to insist they weren't "fatally flawed" at the time, fine, but they certainly are relative to 2020 even by your definition.
2018, 16% still used Subversion. https://insights.stackoverflow.com/survey/2018#development-p...
But that's the hope, that we have something better by 2040, that makes today's Git look as outdated as CVS/Subversion looks to us now.
But if your Subversion example is one of those fatal flaws, then I just link to his example of Git usage that probably happens more often than we want to admit https://xkcd.com/1597/
Using your tool wrong is not a fatal flaw of the tool.
From what I saw, it was git vs hg - which had different philosophies. Hg gave you nice, polished workflow for supported tasks. Git gave you building parts that you can make your own system from. It turned out that enough programmers wanted to build from parts.
SVN required a centralized server and didn't even supported branching, so no.
One can argue about what a branch is, for sure. Subversion branches, Mercurial branches and Git branches all have very different implementations.
But the main workflow of creating a feature branch, developing a feature, then merging it back into the trunk/master, has, as far as I know, always been supported in Subversion.
SVN's official response to branching and tagging was to copy directories around in the repository. Arguably that means it does not support branching, at least according to the concept that has been in place for the last decade.
However, don't let that confuse you. The "svn copy" command creates a new place where you can do independent development, and where afterwards merging back can occur. So it's the same as a branch by that definition.
An "svn copy" is a lightweight copy, with a reference back to the place the copy was made from.
- "svn merge" does its merge by looking at what commits have been made at the original location (since the copy), and the commits to the copy.
- "git merge" does its merge by looking at what commits have been made on the original branch (since the new branch was created) and the commits on the new branch.
So "svn merge" and "git merge" act in the same manner. (Of course there are differences in the algorithm, but I wish to refute the point that Subversion "does not support branching [at all]".)
And if you never use "svn merge" or "git merge", then a branch is just a copy, in either system.
It was not a naming issue. It was a bona fide lack of support for a basic feature. The manual itself states quite clearly that in SVN land you create a branch by copying around the entire working directory within your repository. SVN's manual is quite clear on how SVN actually tracks the state of a file system, and not the state of a source code tree. By copying the working directory around the repository you're creating new revisions of your file system. That' it. Just because you can diff two directories it does not mean copying a directory around in a file system is a branch. And nor is it a tag.
It did have problems with occasional inexplicable tree conflicts, which I do not miss. But that's the same as the pain of trying to rebase in git when things have diverged and it just starts spewing repeated conflicts for each and every commit. In either case, the easiest thing to do is make a fresh branch, patch your changes over, and go from there vs. trying to reintegrate the broken branch.
The other big thing was free cloud hosting for the repos.
Neither of these are unique to Git, obviously, or even introduced by it. It just happened to be the right combination of features and speed at the right time to become the winner of the popularity contest. Kinda like C.
And, just as C is still around, for all its numerous horrible quirks, Git will likely be around for a long time as well.
From where I sit, Git has a couple obvious flaws, and I expect its successor will be the one that fixes one of them. The most obvious (and probably easiest) is the monorepo/polyrepo dichotomy.
[1] https://devblogs.microsoft.com/bharry/the-largest-git-repo-o...
We have 600 devs and face these problems. I can assure you we sure as hell dont have the resources spare to reroll git. We're way too busy rerolling everything else.
The monorepo/polyrepo discussion exists apart from your choice of version control system and has little to do with Git, as far as I can tell
Merge conflicts.
If the conflict is huge, rebasing can help you by "playing" the commits from one branch one at a time so the conflicts are smaller / easier to fix.
At a previous job, a team was forced to use checkout-style VCS due to their manager's unfounded fear of merge conflicts; I couldn't go in that office without hearing one developer shout to another: "Hey, can you finish up and check in that file so I can get started on my changes?"
This consideration is actually irrelevant to locking non-mergeable binary files. It doesn't matter what branch we're on or where the file is located, only that you and I both want to edit the logo. Eventually, either your version must be based on mine, or mine based on yours, since they will be merged.
So it's probably better not to have that file in Git, since it doesn't support the workflow around which Git is based.
It's actually right to store your design documents in Google Docs or a wiki and your code in Git, rather than everything in Git.
It is easy to have one filestore to rule them all and in the darkness bind them, but if you want to do different things with them, you have to do different things with them. I'm not sure that it's possible to unify text file and binary doc based workflows, but it seems we don't have to worry because users automatically use the best tool for the job and it's only hackers who tie themselves in knots trying to make git do everything.
I'd rather not lock files.
I don't think I need to sell the value of DVCS over VCS, but what seems to get lost is that buys you a certain amount of essential complexity, expressed in the CAP theorem and its consequences.
We discussed this deeply on the Fossil forum: https://www.fossil-scm.org/forum/forumpost/2afc32b1ab
We came to no easy answers, because there aren't any. You only get a choice of which problems to accept.
The alternate method is that locking a file marks you as an interested party to a merge, allowing you to review the correctness of a merge.
This would be purely to avoid changes being lost during merges.
Large files and long histories hinder its total dominance in the game and art industries. Because of git's shortcomings polyrepo is a near necessity not simply a stylistic choice. LFS is a bolt on solution that could/should have better support.
I'm intrigued by this claim. I've come to the opposite conclusion - that monorepo is near necessity with git because there's no tools for branching/rebasing multiple repos at once.
I would use it for storing and syncing libraries of large images for my photography, if it were feasible.
Which one you should use depends on which downsides are less impactful for your use case.
As an optimization, when it packs several objects together in a pack file, it can store objects as a delta to other (possibly unrelated) objects; there's a whole set of heuristics used to choose which objects to delta against, like having the same file name. And yes, one of these heuristics does have an effect similar to "reverse chronological order"; see https://github.com/git/git/blob/master/Documentation/technic... for the details.
Like gitlabs own web ide that seems to ignore LFS rules.
It would also be nice to have a repo that isn't language-agnostic. It's too easy to track non-semantic changes, like white space.
In my opinion this is a problem with programming languages rather than version control. Namely we mix presentation and representation when using text as our source code. In the case of whitespace we have an infinite number of syntactic presentations which all correspond to the same semantic representation. Tooling has been created to try to deal with this such as code formatters which canonicalize the syntactic presentation for other tools. Git itself even has to deal with this because of platform differences, i.e. LF and CRLF.
I loathe this about git. It has caused my team members so many problems, like images marked as changed.
git should either be dumb about content or smart, not secretly in between.
You are in the minority. It's such a ubiquitous experience, a running joke in the industry. Saying a tool is useful and powerful, is fine and good. That sentiment has nothing to do with usability.
After I had to unfuck a repository for the n-th time, I trialed a switch to Mercurial, we switched shortly after. I can count on one hand how many times I've had to intervene in the last few years.
And they do occasionally put some new stuff in that helps. Like the recent version which adds new commands to split out the two completely different uses for `git checkout` (making/switching branches and reverting files).
i would love to be able to use a patch-based VC.
unfortunately there are no patch-theory based VCS’s with a practical level of usability. what git was to monotone, X is to darcs/pijul, where X hasn’t been created yet.
"Need a centralized server" wasn't specific to one (pre-DVCS) system, either.
Subversion did this better even before Git existed.
Git is completely oblivious to moving code from one file to another. Git blame will never show you the original commit if you just relocated a method to another file. Due to this, refactoring often put additional hurdles into exploring the code history.
Personally I don't see git's problems with large binary files and tens of millions of commits as being major issues. Those two alone are way less valuable than git's ecosystem and mindshare.
Otherwise it won't just be new languages which suffer, but users of supported languages will suffer when there's an upgrade.
Arbitrary diff/merge in Git is a great example of the Turing Tar-Pit. It's possible, but prohibitively inefficient for many things I want to do. You can't add your own types, index, or query optimizations.
Today, if I want to store data for my application, I have a choice between good support for rich object types and connections (e.g., Postgres), or good support for history and merging (e.g., Git). There's no one system that provides both.
I could see myself agreeing to that.
The slowness of destroying a whole database and then recreate it when checking out should be something you can handle by relying on the diff to generate a series of delete commands and a series of insert commands.
But yeah, I guess committing will be slow if you have a lot of data to export. For the time being, it's a trade off to be made.
[I might consider testing this with my current database project. But I'm using SQLite so I guess that implies a lot less data than Postgres.]
A modern but still centralized VCS like Subversion or Perforce is what you get if you first add networking (CVS) and then atomic commits. Without atomic commits you are pretty much forced to keep a centralized server, and Subversion didn't try to change the server model after adding atomic commits.
DVCS instead is what you get if you start with local revision tracking like RCS, and add atomic commits before networking. Now the network protocol can work at the commit level and is much more amenable to distributed development.
Check out git-diff(1) and --diff-algorithm. --anchored is the one I find the neatest.
I like the way you put this. In case anyone's interested in brainstorming I'm dabbling in this problem with a thing called TreeBase (https://jtree.treenotation.org/treeBase/). It's still a toy at this point, but it stores richly typed data as plain text files to leverage git for history and merging and then can use SQLite (or others) for querying and analysis. A very simple database in the wild looks like this: https://github.com/treenotation/jtree/tree/master/treeBase/p...
Imagine instead of that were available as a sort of materialized view.
I don’t understand what you mean by this, can you provide some more detail?
Saying just parse it every time is denying that there are very real costs associated with hat decision.
Envy does versioning of classes and methods, and you can programmatically access to the model.
It allowed us to build tools around the VCS. For example, we had tools to merge multiple feature branches and resolve conflicts automatically. We also used the same tools to produce migration scripts for our database (GemStone). That was 18yrs ago! and today sounds irreal.
You can build tools on top of git, but the versioning “unit” gets in the way. (e.g imagine the possibility to encode refactorings in your change history and reapply or rollback them).
I’m not trying to criticize git. I think it is the best file based DVCS. My point is that many dev tools that we use today are extremely rudimentary, because the lack of good abstractions. And I don’t think that git provides a good model to build those abstractions on top of it.
If that was then extended into the version control system that'd be even better. Oh yes.
But getting a new language into these things would probably be a nightmare.
https://github.com/unisonweb/unison
Most language ASTs also don't encode useful-to-the-programmer but useless-to-the-compiler information like comments and whitespace. There's been good progress in that (the Roslyn AST system has some neat features), but in practice an AST is always intended more for the compiler than the user/source writer. This also is reflected often in speed, a lot of languages have a relatively slow AST generation (which would add sometimes very noticeable wall clock time to commit time, depending of course on language and hardware).
Plus, of course, all the usual bits that ASTs are extremely varied among themselves (some are weirder DAG shapes than trees, for instance).
An experiment I ran was to the use the "next step down" from full AST which is your basic tokenizer / syntax highlighter. Those designed to deal well with malformed/unfinished/work-in-progress input, and to do it very quickly. Years back I built a simple example diff tool that can do token-based diffs for any language Python's commonly used syntax highlighter Pygments supports. [1] In my experiments it created some really nice character-based diffs that seemed "smart" like you might want from an AST-like approach but just by doing the dumb thing of aligning diff changes to syntax highlighting token boundaries.
You could even use it/something like it/something based on it today as your diff tool in git if you wanted, with the hardest part configuring it for which language to use for what file. (I never did do that though, partly because the DVCS I experimented with this for didn't have a pluggable diff system like git does, nor did it support character based unidiff as a storage format which the experiment was partly to prove both ideas could be useful.)
[1] https://github.com/WorldMaker/tokdiff
Also: > The responsibility of not mixing super and sub-project code in commits lies with you.
Is a lie. The responsibility of not mixing code lies with every member of the team. Does the author work alone?
The successor might very well be git 3.0, though.
Isn't that being addressed in Git with the partial clone functionality?
- Interoperate with the major player(s), currently Git and in many places unfortunately still Subversion. svn2git probably did more for Git adoption than any other feature or tool, because it allowed a fairly painless transition without losing information.
- Solve at least one big problem with the current contenders. Git made it possible to run VCS without a separate server program and sped up VCS operations massively. Both of those were huge. Looking at the Fossil home page[1] it does have some features I personally have wanted in VCSes, such as integrated "bug tracking, wiki, forum, and technotes," but the devil is very much in the details of how that actually works (How easy is it to write your own custom frontend or add business-critical bug tracking fields, for example?), and it's not like we don't already have good bug trackers, wikis etc.
Just as a tangent, some issues (but not necessarily major or fundamental, depending on who you ask) with Git as it works right now:
- Does not use cryptographically secure hashes, and has no clear migration path to a different hashing mechanism.
- Git Annex is not yet built in.
- The command line is complex, including many niche subcommands and tons of rarely used options.
- The command line is inconsistent, such as `git rm` vs `git branch --delete` vs `git remote remove`.
- It is based on a less than ideal theoretical model of patches[2]. IMO this is the most exciting development in VCSes since Git.
[1] https://fossil-scm.org/
[2] https://pijul.org/model/
Which is a better choice than Git for most projects, to be honest.
No, people will happily destroy history. Git is just a tool.
In extremis, Fossil offers the "shun" command to remove improperly-committed artifacts, but even then it's subject to a lot of restrictions. (https://fossil-scm.org/fossil/doc/trunk/www/shunning.wiki)
Tools don't have opinions, people do. Fossil is just inflexible.
You don't want to alter history? Don't do it then. Git supports not altering history just fine.
Tools being more flexible is strictly a good thing. If they are misused, the person that misused them is responsible. It is that simple.
You're confusing a hosting service with being forced to use a centralized repository.
Take GitHub, for example. If git was centralized them you would not have forks, multiple remotes or multihosting, or could even work independently of the remote server.
With Git, you can even set up a repo in a network file system somewhere, or even a USB thumbrive.
Meanwhile SVN supports forking in the sense that it supports copying directories.
You are probably thinking of branches and tags, and those are used similarly in both systems. They are a bit more convenient in git since they are created in constant as opposed to linear time.
Copies in Subversion are only metadata. Since partial clones are native to the system, it is simple to present both branches and tags as file paths. It was likely considered an easy user interface. Everything is a file and all that. In comparison, git users must learn the git object naming scheme, otherwise things can end up very confusing should you have a directory and a branch with the same name.
I would argue that branches are nicer in git because it makes merging so much easier.
No, I was referring to forking projects, in the sense that Git allows for creating brand new and independent repositories that clone the version history up to a point in time and enable to add the origin Git repository as a remote repository.
SVN also does not support branching or tagging, as it actually supports only copying directories around a file system.
Linus seems to have a very different development model than almost all of the rest of us. And if Linus is asynchronous, then so is the codebase he maintains.
If you or I have eight different PRs going it’s because we have eight different promises being made to customers. Rejecting them has real consequences that we feel. For Linux many of those are externalities.
That's pretty subjective. Most engineers I interact with personally know how to use Git but not SVN. I'd argue the opposite is true base on my subjective experience.
Really depends on where/how you're using versioning, team familiarity, etc.
That’s a little too much to explain away with variability.
Atomic commits and file renames.
With time, the standard requirements rise and we get more demanding.
How about a VCS that not only diffs line by line but saves the diffs as editing delta?
Or that understands the syntax of the code and can differentiate a variable renaming from an actual code change?
Or connecting different repositories is hardly solved with Git. Submodules suck, grafting even more so. Or you can't make one repository out of two and keep their Git history intact.
https://github.com/git/git/blob/master/Documentation/technic...
I have some collaboration tools I’d like to write but creating my own edit histories and conflict resolution is daunting.
Commit messages are not versioned.
I can't go back and change the commit message to fix a typo or add something that was missed without changing the history.
This always bothered me somewhat but now that I have non native and native people on the team that can't spell, I feel the pain everyday.
https://www.atlassian.com/git/tutorials/rewriting-history#gi...
Ammending changes the hash of the commit. That doesn't matter if you haven't pushed it yet but if you have, you can't push the change upstream without force-pushing.
And if I want to change a commit message further back in history (regardless of my own or of somebody else), it would branch off at that point from the upstream git history.
What I want is that you could put multiple comments onto the same commit with different timestamps. Like git versions files, it should also allow to version commit messages.
The key point is "your own tooling". Git has a great tool universe but unfortunately you immediately lose many tools if you go beyond what the core offers.
Not necessarily. Fossil's `amend` command works by adding additional information to the repo that the web UI and commands like `fossil info` look at when building up information intended for direct consumption by the user.
In this way, you can edit commit messages, rename branches/tags, add/remove tags, etc. to historical check-ins without breaking the blockchain / Merkle tree of commits.
This allows Fossil to keep all of the historical information about what happened to a given file, commit, ticket, etc. while still allowing a coherent presentation of the current state of affairs to the user.
If that never happens, then it's a pretty strong indication that Git is working perfectly fine -- so why worry about whether or not you can replace it?
In a way, the git monopoly is worse than Windows or x86 or IPv4, because it's not just a piece of technical infrastructure. Its arcane commands and its branching model have infected all of our brains. You can choose a different editor, you can choose a different operating system, but for as long as we all live, we will never escape the fact that "git reset" does a half a dozen confusingly different operations, or that renaming cannot be tracked, or that most users don't fully understand most of the commands they regularly use.
Our plan in Dark (https://darklang.com) is to combine all the different ways that people "branch" (deployment, feature flags, git branches, staging/dev/prod environments) into a single concept. And then we also plan to combine all the ways to "comment" (PRs, slack messages, commit message, code comments) into a single concept.
Not sure if you'd call that a git replacement, but it's a displacement of sorts - the function of git is replaced by non-git.
The first concept we've been calling deployless, and discussed it here: https://medium.com/darklang/how-dark-deploys-code-in-50ms-77...
The second one is just an idea right now, suggestions welcome. The observation is that comments on a particular line of code are spread in as many as a dozen places (a google doc, slack, trello, the code itself, an old version of the same code, PRs on github, comments on commits on github, commit messages, another place in the codebase referencing this one, the docs folder in your repo, another repo that uses this API, your 3rdparty docs on README.io, etc). This is weird and bad, and it must be possible to do better.
How about one system which is a hypermedia system and can hold many different kind of objects. So annotation objects can reference 'code objects' such as types, fields, functions or even blocks directly. Not sure if Dark gives each of these an identity (it should) which will make it possible to refer to them via hard links rather than text snippets. Once you have all code and comment objects in one hypermedia system, creating views from that is about multiple projections of the interconnected objects.
Wouldn't it be great if I can use a query to refer to 'all functions that reference this type' inside some docs? Or list all annotations that reference a function? These could be embedded inside annotations as well. Gtoolkit does something similar.
Perhaps even a 'branch' can be thought of as a subset of the hypermedia graph. E.g. using a code block X2 instead of a code block X1 within the same function.
These replacements were seemless to users. New Ethernet adapters were compatible with at least the previous spec. The Git import/export of Fossil is not seemless at all. It actually adds quite a bit of complexity if you want to introduce it your regular workflow.
> 2. Microsoft's long-term stalwarts Windows and Office are dying...
Citation needed.
> 3. Adobe's having a hard time hanging onto its old market...
Citation needed.
> 4. IPv4 still won't go away...
There is a lot of hardware out there that only works with IPv4. The costs and risks of switching your org's internals, product or services from IPv4 to IPv6 are phenomenally higher than switching your org from Git to Fossil or adding Fossil support to your product or service.
If Fossil is truly superior to Git and people are not switching to it, then there's no hope they'll switch to IPv6. Not until the cost of not switching is greater.
Not at all. I remember how long it took for Cat 5 to replace thinnet.
There was partial protocol compatibility between ethernet and wifi, but the user experience was very different. Some features were lost or degraded (eg speed, reliability, security, configuration complexity) but a pain point (cables) was fixed.
If an easier VCS caught on enough to be used in schools and boot camps, many younger devs would start with that one and just keep using it as they progress in their career. The new VCS wouldn't have to be better or more powerful, just easier to use for the basic stuff that makes up 95% of Git usage, in order to gain traction. That said, Git works perfectly fine and is so entrenched that I don't see it going anywhere.
That problem could be gotten around if there was some enormous pool of potential VCS users who aren't using VCS currently but would if there was a better one. I don't think there is.
I think this is true if you think the target market for VCSs is professional software developers only. But there are other people writing code that might care less about the fact that Git is the standard for professional software development and don't want to pay the cost of learning or using Git: data scientists, scientists in general, new media artists, most high school students, etc. I've had a hard time selling Gitless (https://gitless.com) to undergrad CS students, but it is easy to sell to non-CS students that write code.
That said, GitHub is a big thing so any new VCS probably needs some story for Git-compatibility. Even tools that have built-in version control like Overleaf have some way of synchronizing with Git repos.
Gitless looks great. It looks like you nailed all the main issues I have with git. If you ensure that people never have to use the regular git interface, maybe it'll take off. I hope so!
BTW Daniel was my PhD advisor :-).
Yes, but maybe they care less about using Git compared to another VCS as long as the other VCS interacts with Git and they can put their repo on GitHub. The question is:
If you can use a VCS that is easier to learn/use than Git and that is compatible with Git so that you can put your repo on GitHub if you want to, would you use it? If no, why?
My guess is that most professional programmers would answer "No, because Git is an industry standard and I need to know Git to get a good job", while other people that write code but have no intention on becoming professional programmers are much more likely to answer "Sure, why not!".
If you ask your kids, I'd love to know what they said :)
Gitless looks great. It looks like you nailed all the main issues I have with git. If you ensure that people never have to use the regular git interface, maybe it'll take off. I hope so!
BTW Daniel was my PhD advisor :-).
Thank you!
That's what the person you are responding to is talking about.
UI = User Interface which is the interface between the User and the Program GUI = Graphical User Interface. Same thing, but pretty.
A better Git UI is possible, like Gitless. A GUI that just has the functionality of the Git CLI doesn't solve the problems I care about, like the staging area being unnecessary and complex.
That’s a UI!
Five years after introducing a replacement you could have half a team that never used it at all, just like SVN is now.
I think part of the reason is that SVN never took over from CVS as completely as git took over from everything else. I think another part of the reason is that the number, size and scope of open-source projects exploded since git and Github appeared, and they mostly chose git.
Also relevant: the only project that I recently interacted with that was using SVN is LLVM, and it was faster/easier to use the Github mirror than to use SVN.
It's ironic that Git was popularized in the same era as monorepos, yet Git is a poor fit for monorepos. There have been some attempts to work around this. Google's `repo` command is a wrapper around Git that treats a set of smaller repos like one big one, but it's a (very) leaky abstraction. Microsoft's GVFS is a promising attempt to truly scale Git to giant repos, but it's developed as an addon rather than a core part of Git, and so far it only works on Windows (with macOS support in development). GVFS arguably has the potential to become an ubiquitous part of the Git experience, someday... but it probably won't.
Git also has trouble with large files. The situation is better these days, as most people have seemingly standardized on git-lfs (over its older competitor git-annex), and it works pretty well. Nevertheless, it feels like a hack that "large" files have to be managed using a completely different system from normal files, one which (again) is not a core part of Git.
There exist version control systems that do scale well to large repos and large files, but all the ones I've heard of have other disadvantages compared to Git. For example, they're not decentralized, or they're not as lightning-fast as Git is in smaller repos, or they're harder to use. That's why I think there's room for a future competitor!
(Fossil is not that competitor. From what I've heard, it neither scales well nor matches Git in performance for small repos, unfortunately.)
Git's flaws are primarily in usability/UX. But I think for its purpose, functionality is far more important than a perfect UX. I'm perfectly happy knowing I might have to Google how to do something in Git as long as I can feel confident that Git will have the power to do whatever it is I'm trying to do. A competitor would need to do what git does as well as git does it, with a UX that is not just marginally better but categorically better, to unseat git. (Marginally better isn't strong enough to overcome incumbent use cases)
And for the record: I think git-lfs issues are primarily usability issues, and tech improvements. The tech enhancements will be solved if there's enough desire, and as I mentioned the usability problems are more annoyances than actual problems.
I would love a layer over Git to handle workflow issues related to multi-repo projects
I constantly run into git scalability issues as an individual. I don't use any of the UI clients because they all fail hard on mostly-code git repositories. I abandoned my VisualRust port in part because the mere 100MB of mingw binaries involved for that meant it was using github LFS, which meant CI was hitting github quota limits, and as I wasn't part of the organization - nevermind an admin with billing rights - I couldn't even pay to up said quota limits paying out of pocket myself even if I wanted to.
I'm not going to inflict git's command line experience - which confounds and confuses even seasoned programmers - on any of the less technical artists that might be employed at a typical gamedev shop, even if git might be able to scale acceptably if locally self-hosted at a single-digit employee shop.
A few dozen or hundred employees? Forget it. Use perforce, even though it costs $$$, is far from perfect, and also has plenty of scaling issues eventually.
That one of - if not the - most popular tool to solve said git scalability problems, also has scalability problems in practice, is both ironic - and absolutely a problem with the git ecosystem. To be pithy - "Even the workarounds don't work."
"Technically", you might say, "that specific symptom with git lfs, and that service provider, isn't the fault of git the command line tool, nor the git protocol". And you would be technically correct - which is the best kind of correct.
But I don't think we're referring to either of those particularly specific things with "Git" when we ask the article's question of "Is Git Irreplacable?". I'm already the weirdo for using git the command line tool - most of my peers use alternative git UI clients, and I don't mean gitk. The git protocol is routinely eschewed in favor of zips or tarballs over HTTPS, Dropbox, Sneakernet, you name it - and is invisible enough to not be worth complaining about to pretty much every developer who isn't actively working on the backend of a git client or server. Not to mention it's been extended/replaced with incremental improvements over the years already.
So I'm using a slightly broader definition of "git", inclusive of the wider ecosystem, that allows me to credit it for the alternative UI clients that do exist, rather than laughing off the question at face value - as something that has already been replaced.
Github the company is not interested in providing you (or anyone else) with free storage for arbitrary data. You were unable to pay for the storage options they do provide because you did not have admin rights to the github account you wanted to work with.
None of this is a problem with git, be it GUI git clients or command line ones.
This isn’t just "technically correct". It’s the "a commercial company doesn’t have to provide you with a service if they don’t want to" kind of correct.
All the commercial service providers recommend keeping total repository sizes <1GB or so, and I hear nothing but performance complaints and how much they miss perforce from those who foolishly exceed those limits, even when self hosting on solid hardware - which is 100% the fault, or at least limitation, of git - I believe you'll agree.
LFS is a suggested alternative by several commercial service providers, not just one, and seems to be one of the least horrible options with git. You're certainly not suggesting any better alternatives, and I really wish you would, because I would love for them to exist. This results in a second auth system on top of my regular git credentials, recentralization that defeats most of the point of using a DVCS in the first place, and requires a second set of parallel commands to learn, use, and remember. I got tired enough of explaining to others why you have a broken checkout when you clone an LFS repository before installing the LFS extension, that I wrote a FAQ entry somewhere that I could link people. If you don't think these are problems with "git", we must simply agree to disagree, for there will be no reconciling of viewpoints.
When I first hit the quota limits, I tried to setup caching. Failing that, I tried setting up a second LFS server and having CI pull blobs from that first when pulling simple incremental commits not touching said blobs. Details escape me this long after the fact - I might've tried to redirect LFS queries to gitlab? After a couple hours of failing to get anywhere with either despite combing through the docs and trying things that looked like they should've worked, then I tried to pay github more money - on top of my existing monthly subscription - as an ugly business-level kludge to solve a technical issue of using more bandwidth than should really have been necessary. When that too failed... now you want to pin the whole problem on github? I must disagree. We can't pin it on the CI provider either - I had trouble convincing git to use an alternative LFS server for globs when fetching upstream, even when testing locally.
I've tried gitlab. I've got a bitbucket account and plenty of tales of people trying to scale git on that. I've even got some Microsoft hosted git repositories somewhere. None of them magically scale well. In fact, so far in my experience, github has scaled the least poorly.
> Github the company is not interested in providing you (or anyone else) with free storage for arbitrary data.
I pay github, and tried to pay github more, and still had trouble. Dispense with this "free storage" strawman.
> You were unable to pay for the storage options they do provide because you did not have admin rights to the github account you wanted to work with.
To be clear - I was also unable to pay to increase LFS storage on my fork, because they still counted against the original repository. Is this specific workaround for a workaround for a workaround failing, github's fault? Yes. When git and git lfs both failed to solve the problem, github also failed to solve the problem. Don't overgeneralize the one ancedote of a failed github-specific solution, from a whole list of git problems, to being the whole problem and answer and it all being github's fault.
> None of this is a problem with git, be it GUI git clients or command line ones.
My git gui complaints are a separate issue, which I apparently shouldn't merely summarize for this discussion.
Clone https://github.com/rust-lang/rust and run your git GUI client of choice on it. git and gitk (ugly, buggy, and featureless though it may be) handle it OK. Source Tree hangs/pauses frequently enough I...
If my computer caught fire and exploded due to poor electrical design, you wouldn't say "nothing about your problems had anything to do with your computer and everything to do with the specific company that provided your pencils" when in my growing list of fustrations I offhandedly mentioned breaking a pencil tip after resorting to that, what with the whole computer being unavailable and all. That would be weird.
Even if we did hyper focus on that pencil - pretty much every pencil manufacturer is giving me roughly the same product, and the fundamental problem of "pencils break if you grip them too hard" isn't company specific. It's more of a general problem with pencils.
Github gave me a hard quota error. Maybe Gitlab would just 500 on me, or soft throttle me to heck to the point where CI times out. Maybe Bitbucket's anti-abuse measures would have taken action and I'd have been required to contact customer support to explain and apologize to get unbanned. git lfs's fundamental problem of being difficult to configure to scale via caching or distribute via mirroring isn't company specific. It's more of a general problem with git lfs. Caching and mirroring are strategies nearly as old as the internet for distribution - git lfs should be better about using them.
It would've turned github's hard quota error into a non-event, non-issue, non-problem - just like they are with core git. Alternatively, core git should be better about scaling. Or, as a distant third alternative, I could suggest a business solution to a technical problem - GitHub should be better about letting me pay them to waste their bandwidth. Then I could workaround git's poor scaling for a little bit more, for a bit longer.
A major limitation of git is how it deals with many "big" (~10Mb) binary files (3D models, textures, sounds, etc.).
We ended up developing our own layer over git, and we're very happy ; even git-lfs can't provide similar benefits. This technique seems to be commonplace for game studios (e.g Naughty Dog, Bungee), so certainly git has room for improvement here.
I'd argue if it's the later, that git was never the right choice to begin with. You don't really want to record a full 10MB of data every time you change one pixel in your texture or one blip in your sound, right?
So I don't know if this is a "major limitation" of git per se. Not saying there's a better solution off-the-shelf (you're obviously happy with your home grown). But this was probably never a realistic use for git in the first place.
> You don't really want to record a full 10MB of data every time you change one pixel in your texture or one blip in your sound, right?
Actual changes to content in a gamedev studio are very unlikely to be as small as a single pixel. Changes to source code are unlikely to be as small as a single character either. And we definitely want a record of that 10MB.
We're willing to sacrifice some of our CI build history. Maybe only keeping ~weekly archives, or milestone/QAed builds after awhile, of dozens or hundreds of GB - and maybe eventually getting rid of some of the really old ones eventually. Having an exact binary copy of a build a bug was reported against can be incredibly useful.
http://www.daemonology.net/bsdiff/
https://www.chromium.org/developers/design-documents/softwar...
Sure, immutable build artifacts can be invaluable -- but aren't they also an orthogonal concern?
One person's immutable build artifact is another person's vendored build input.
It's common to vendor third party libraries by uploading their immutable build artifacts (.dll, .so, .a, .lib, etc.) into your VCS, handling distribution, and keeping track of which versions were used for any given build. It makes a lot of sense if those third party libraries are slow to build, rarely modified, and/or closed source - no sense wasting dev time forcing them to rebuild it all from scratch.
The next logical step is to have a build server auto-upload said immutable build artifacts into your VCS, for those third party libraries that you do have source code for, when your VCS copy of said source is modified. Much more secure and reproducable than having random devs do it.
And hey, if your build servers are already uploading build artifacts to VCS for third party libraries, why not do so for your own first party build artifacts too? Tools devs spending most of their time in C# probably don't need to spend hours rebuilding the accompanying C++ engine it interoperates with from scratch, for example, so why not "vendor" the engine to improve their iteration times?
This can lead to dozens of gigs of mostly identical immutable build artifacts reuploaded into your VCS several times per day, with QA testing and then integrating those build artifacts into other branches on top of that. The occasional 10MB png is no longer noticable by comparison.
But I'm totally willing to fault git for failing to optimize enough to handle the manual commit cadence of source game assets though. Because that's not just a tertiary use case - frequently for coworkers it's their primary use case. The end result is I mostly only use git for personal hobby stuff, where it's a secondary use case and my assets are sufficiently small as to not cause problems.
Build artifact caching is a different problem from source control, with very different requirements:
1. As you mentioned, the artifacts tend to get huge.
2. The cache needs to be easy to bypass. From your example, it needs to be easy for the C++ engine devs to do builds like "the game but with the new engine" to test out their changes.
3. The cache needs to be precise, so you don't end up with mystery errors once it finally does trigger, or people wondering why their changes don't seem to apply.
4. The builds need to be exactly reproducible, so you don't end up with some critical package that only Steve Who Left 5 Years Ago could build (or Jenkins Node 3 That Just Suffered A Critical HDD Failure).
Git either doesn't care about or fails spectacularly for each of those points. In particular, #3 will be very confusing since there will be a delay between the code push and the related build push.
Nix[0] solves #2 and #3 by caching build artifacts (both locally and remotely[1][2][3]) based on code hashes and a dependency DAG (for each subproject or build artifact, so changing subproject X won't trigger a rebuild of unrelated subproject Y, but will rebuild Z that depends on X). It helps with #4 by performing all builds in an isolated sandbox.
#1 is solved by evicting old artifacts, which is safe as long as you trust #4. If the old artifact is needed again then it will be rebuilt for you transparently. Currently this is done by evicting the oldest artifacts first, but it could be an interesting project to add a cost/benefit bias here (how long did it take to build this artifact, vs the amount of space it consumes?).
[0]: https://builtwithnix.org/
[1]: https://nixos.wiki/wiki/Binary_Cache
[2]: https://nixos.org/nix/manual/#sec-sharing-packages
[3]: https://cachix.org/
> 1. As you mentioned, the artifacts tend to get huge.
This, admittedly, is more common with build artifacts. That said, I've hit quota limits with autogenerated binding code on crates.io, with several hundred megs of code still being in the double digits when better compressed by cargo than I can figure out how to compress with 7-zip.
And that's a small single person hobby project, not a google monorepository.
> 2. The cache needs to be easy to bypass
I need to bypass locally vendored source code frequently as well, to test upstream patches etc.
> 3. The cache needs to be precise, so you don't end up with mystery errors once it finally does trigger, or people wondering why their changes don't seem to apply.
Also entirely true of source code.
> 4. The builds need to be exactly reproducible, so you don't end up with some critical package that only Steve Who Left 5 Years Ago could build (or Jenkins Node 3 That Just Suffered A Critical HDD Failure).
Enshrining built libs in VCS is an alternative tackling of the problem. You might not be able to reproduce that exact build bit-for-bit thanks to who knows what minor compiler updates have been forced upon you, but at least you'll have the immutable original to reproduce bugs against.
> In particular, #3 will be very confusing since there will be a delay between the code push and the related build push.
It's already extremely common - in the name of build stability, including with git - to protect a branch from direct push, and have CI generate and delay committing a merge until it's verified the build goes green. By wonderful coincidence, this is also well after CI has finished building those artifacts - in fact, it's been running tests against those artifacts - so it can atomically commit the source merge + binaries of said source merge all at once. No delay between the two.
There are some caveats - gathering the binaries can be a pain for some CI systems, or perhaps your build farm is underfunded and can only reasonably build a subset of your build matrix before merging. Or perhaps the person setting it up didn't think it through and has set things up such that code reaches a branch that uses VCS libs before the built libs reach the same spot in VCS - I'll admit I've experienced that, and it's horrible.
Nix, Incredibuild, etc. are wonderful alternatives to tackle the problem from a different angle though.
Ideally, yes, why wouldn't I? I want to capture the exact state of the thing at each change.
I totally get the use case of saving each iteration of that 10MB file _somewhere_. But expecting git to do that job is not the right level of expectation, was my main point.
When I have worked with binaries like that described, I will place a URI reference to a file hash and have something that knows how to resolve it. A file store (think S3 or whatever) that has files named: texture1.dat-[sha1] and change the reference to the file in the source. e.g. a "poor man's" version control by way of file naming conventions. Does this approach work in your world?
If somebody comes up with something that matches Git's strengths and also handles binaries and biggies much, much better then they could definitely topple Git with it. It'd take time for the word to spread, the tools to mature and the hosting to appear, but I can definitely see it happening.
I think most people know that Git isn't perfect, but it's also the case that coming up with anything better is an extremely difficult task. If it wasn't, someone would have already done it. It's not like people haven't been trying.
Diffing a PSD as a binary is impossible - whereas proper asset management tools will deconstruct the PSD’s format to make for a human-readable diff (e.g. added/removed layers, properties, etc).
I would say that the sole thing git was developed for, the Linux Kernel, is (starting to be) painful to work with when using git.
Honestly asking.. Do you speak from some level of authority that the Linux kernel is stretching the boundaries of git? Or are you just saying that more speculatively? What is the painful part?
The magic sweet spot might be the fact that most projects to not need to be distributed. This is where a lot of complexity is derived.
So without all those extra concerns - and - a more elegant UI framework (i.e. rational commands) - and possibly something that scales a little better. That's enough mojo to unseat git for a lot of things.
I recollect that for Windows (which also uses git), MS have actually extended git with "Git Virtual File System" rather than replace it[1]. But I do agree that broadly, not everyone needs the scale.
[1] https://devblogs.microsoft.com/bharry/the-largest-git-repo-o...
- http://www.paulgraham.com/hamming.html
You can get around branching-is-bad by changing your workflows a bit, but you can't get around the bad merges: over time it's like death by a thousand papercuts.
As a Google employee I use hg every day, even though it's not required. (Some teams at Google do mandate its use, but these are few and far between.) I don't use branches, but I use bookmarks. I didn't notice any merges that really ought to be performed automatically but were not; in any case I use Meld to resolve merge conflicts and it's easy enough to do occasionally.
I'm also not certain that the "fundamental design issues" with git are truly fundamental to its design. For example, partial clones and sparse checkouts are seeing increasing support in recent versions of git — and those are really all you need.
What bothers you in particular?
If you want the wacky and unreliable git branching you can use hg bookmarks.
This page has been viewed 230 thousand (!!) times. Because git is so easy and elegant that it lies to you what branches exist on the remote.
It is not even funny any more how bad this is.
Normally mercurial stops when there are conflicts it cannot resolve reliably. In that cases, have a try at kdiff3: it handles hairy merges quite well. In a lot of cases even automatically (and correctly).
There is always meld, but I'd say kdiff3 is superior wrt merge conflict resolution.
Mercurial has 3 ways of doing branching:
- bookmarks: these are like git branches, a pointer to a revision
- branches: when you are in a branch, all commits are permanently affixed with that branch name. Less flexible than bookmarks (and therefore git branches) but good for traceability
- heads: unlike with git, a branch name can refer to several actual branches, it usually happens when you are pulling from a central repository, but you can create them yourself if you need some kind of anonymous branching. These can be pushed but it is not recommended.
Git only has the first option.
The way central repository are managed is also a bit different even if the fundamentals are the same. Git has the "origin" namespace to distinguish remote branches from local branches. Mercurial uses a "phase" which can be "public" (in remote), "private" (local only, will become "public" after a push) and "secret" (like "private", but will not be pushed and therefore will not become "public"). So if you are not synchronized with the remote, in git you will have two branches: origin/my_branch and my_branch, in mercurial, you will have two branches named my_branch, one public, one private. That's essentially the same thing, presented differently.
In the end, they are fundamentally the same. The feel is different though. Git is flexible, and gives you plenty of tools to keep things nice and clean when working with large, distributed project. As expected for something designed for the Linux kernel. Mercurial focuses on preserving history, including the history of your mistakes, and I feel it is better suited for managed teams than a loosely connected community.
* Pushing them is (slightly) more annoying than pushing git branches — you need a separate command, whereas `git push` just does the right thing by default * Deleting them doesn't delete the corresponding commits * There is only one global namespace shared across all remotes
That's interesting. In your examples isn't it fast because monorepos are network-based, as in, you only fetch what you need when you need it?
Also reminded me of discussions around CPython's startup time and how one use case where milliseconds matter is in small cli utilities such as Mercurial.
The entire repo is stored on a networked file system. So essentially every file operation is remote. That doesn't actually contribute to much slowness because when I didn't use hg, operations were noticeably faster.
Nowadays, a new computer means a git clone (or just plain copy-paste) of a USB stick from the old one. This seems like it's a single feature of git that could be written, but if you told me "there's something that works better for large, twenty year old repos", I'd probably take that.
I don't know how Linux survives, but maybe it's just that you only rarely git clone your large repos. (Or maybe it's just that intercontinental internet is less reliable than intracontinental, so that if you're in the US it's a non issue.)
It’s only huge megacorps that need larger scale things like GVFS.
As for large files, that is not what Git is for. Git is for source code. Much like how you don’t put large files in your RDBMS, you should not be putting them in your SCM either.
You can still do it if you want, but you might be better served using https://git-lfs.github.com/ or using another system designed for that purpose.
Could you please provide a link to it? I’m very interested in seeing this command, but ironically it’s not a name that’s easy to google for.
Edit: I was very wrong, searching for “google repo command” displayed https://gerrit.googlesource.com/git-repo as the very first result.
One thing is code review. There is no code review in Git.
What I expect in 2020 is that I should be able to specify reviewers for the commit (which I pick out of a list of people who can approve it). These people should be able to leave comments on the commit. I should be able to both respond to comments and modify the code before the commit gets checked in. The history of the comments and changes should be maintained.
There is nothing in Git that supports this flow in a natural way.
A replacement for (or evolution of) Git can be a tool that would support this code review flow from the get-go.
And yes, there are external tools for code review. But that all should be a part of version control.
The point about code review still stands.
Separately, I know many including myself who would love for code review comments to more seamlessly be integrated into the code browsing experience.
You can keep the banter with the code and the go-no-go decision separate, even external. But post mootems have worked better when someone realized that one of the team repeatedly calls out a class of errors that bite us later and they’re being ignored. You have the ability to prevent this error. Wise up or that person will decide rightfully that we are a bunch of clowns and leave.
Code review comments really should live next to the code, and proper editor integration goes a really long way. Traditional review processes make “just fixing the code” have a very high inertia.
[1] https://blog.janestreet.com/ironing-out-your-release-process...
The version control should keep the code history, not the paperwork history.
What I think you're looking for could, however, use git as a platform for that. That's what GitHub, GitLab and the likes do using a web interface and there's enough extensibility and power on git for command-line or desktop tools to do it.
The internals of git are very much akin to a filesystem, by the way, and the plumbing gives you more enough access to use that in creative ways.
From the top of my head, maybe a system like this could automatically generate tags for code reviews, use merges and branching for answering to these reviews, and hooks for notifying interested parts on particular areas of code. All while the messages and reviews themselves travel on another data layer, which references git but does not mingle with it.
This separation (and even tag cleaning, for example), would be specially useful on huge distributed projects where a company or small team may have whatever development process it needs internally and sharing only the results without having to completely hide the code history.
If all we cared about was code history, a super pure "version control" system would have one trunk, no branches, and a sequentially increasing version number with no commit messages or author information.
But if you can annotate an entire commit with a descriptive message, why not annotate a specific hunk of code changes (comments), or the discussion over those changes in a specific version?
Environments and development practices may change, but the code review that led to a particular code decision at some point in history are a valid representation of the context at that time in history.
Don't get me wrong, I'm not lacking these features in Git, and am happy to get them from other platforms like Github, but I think the comment you're responding to is astute that one can imagine a git replacement that incorporates code review functionality as a first-tier feature.
https://git-scm.com/docs/git-notes
It's very weird to me that you would use reviewer as your example because that's pretty much the exact point of the signoff feature.
I don't agree. The responsibility of a version control system is to manage the changes made to the source code. The paperwork that goes with it is an entirely different, and separate, responsibility, just like ticketing systems and keeping track of tasks, epics, sprints, etc.
> If all we cared about was code history, a super pure "version control" system would have one trunk, no branches, and a sequentially increasing version number with no commit messages or author information.
That doesn't make sense because you're ignoring the fact that branches are used to host versions that are being developed independently at a specific moment in time, and merges are used to finally join contributions when they are ready to be added to the main branch. If you look at single branches and ignore the work being developed in any other branch, the history is as linear as you expected it to be.
> But if you can annotate an entire commit with a descriptive message
But you can.
At which point you get code review that is logged entirely in the history of the repo. Which is a very useful feature, and a huge value proposition of GitHub over raw git - git blame gives you the commit, but GitHub will also gives you the pull request for that commit, and you can go and look at the comments there to understand why it was done the way it was.
A difference between a commit message and a code comment/thread is that a commit message is more authoritative.
(Probably some combination of nobody asking for it, and in some cases, wanting to keep your code review process locked in to the tool.)
Perhaps every project might have a branch called "issues" or "reviews", etc. Not sure the best setup but generally the more I can do in my code editors the faster I can work.
Has anyone found good work patterns like that?
There are many different code-review workflows, and for teams that do full-time pair-programming, code review happens in real-time as the code is written.
Trying to bake support for all of that into something that is also a good VCS sounds like a recipe for one of those clock-birdfeeder-machete-flashlight-massager tools that you used to see on the back pages of outdoor magazines in the 90s.
This is exactly the issue with it: its author(s) lament that git has not all of these things integrated, but they are ignoring that this is exactly why fossil cannot gain a wider adoption.
It has opinions about how the team and the project should be run / managed / documented. This is not for the tool dev to have these opinions
Yes there is. You can block pushing directly to a branch of the repository and demand that things can only be merged through pull requests. You code review the pull requests.
This is very well supported by Atlassian's git tooling, for example. It does everything you mention. It's true that the comments don't become part of the git repository itself, but I'm not convinced they should be.
I expect that git's eventual replacement will initially boast compatibility with existing git repos.
[1] https://trends.google.com/trends/explore?date=all&q=%2Fm%2F0...
[2] https://en.wikipedia.org/wiki/Technology_adoption_life_cycle
Whatever code is in your branch is what it is. And then there's a layer of magic on top of it all, maybe a UI or command line tool or both, just to be able to easily rewind time. Either per file, per line of code, or folder, or just a folder but not recursive, or the entire branch.
Think of Apple's Time Machine, it should be that simple.
Honestly, I rarely–if ever–read commit messages to begin with. The way I navigate older code is never done by searching for commit messages. That's not reliable.
Instead I simply go to a point in time where I think the thing I'm looking for might be. I'll look at the code, recognise its state, and continue the search up or down the timeline.
And that would cover the needs of most projects I'd say. And it would save us a shitload of time.
Hell, I've been working with git for almost a decade now. I never needed to rebase or merge things, until I recently did need to do it. It's too arcane to make intuitive sense, to me anyway.
Git is good, I'd just welcome a breath of fresh air...
Versioned files systems, including Apple's Time Machine and Dropbox's version history, have no extra UI to save versions. A small, short-lived project with at most a few collaborators (eg, working on a small scientific paper) might find those more useful than git or other VCS.
I would not mind something simple. I find the arguments, flags, whatever in git to be rather opaque. That, coupled with the lack of need for its features, have not thrilled me.
It seems to be pretty effective if you have tons of programmers working on a single project, but at the bottom end of the scale, I find it baffling.
(I know, technically, altering commits creates new commits in Git, but it comes to the same thing.)
As a heads up if you're stuck on cvs, svn or other stupid vcs due to "old-codgers" in your office, mercurial has a shallower learing curve and easier ui to get the same job done which may make it easier to switch.
Either git or mercurial, who cares? Either of them until there's something better. Never deal with SVN & CVS branches and merges again. Feel the immediate team productivity boost which will pay for the initial learning curve costs by day 2. Seriously.
Unfortunately, you have to settle on some workflow for that. And this settling will take some experience and discussion. So "day 2" sounds very overambitious to me.
There are definitely times you want to store the full history forever, but it would be nice to have a DVCS that gave other options.
/insert Michael Cain playing Homer sound: Buh-oh./
Those who forget (or rebase away) history are doomed to repeat it.
But I don’t like how difficult databases make it to represent graphs of data. To pull a subtree cheaply, you need a graph. not unlike SVN’s data structure.
I like this kind of untroubled minimalism, and so far have never encountered a reason not to run every personal project on Fossil. The real world will occasionally force me onto Git territory, but I can't really say I have ever enjoyed the experience.
Git is here to stay for any foreseeable future, of course. And while I do understand points often made about the benefit of one de facto standard to rule them all, monolithic dominance always tends to unsettle me. My SE friends in general simply use 'Git' as a given synonym for 'version control'. And it does annoy and somewhat worry me that they've never even heard of Fossil until I roll out my sermon.