Git Sucks, What Am I Doing Wrong?

19 points by JacKTrocinskI ↗ HN
My employer recently made a companywide switch from SVN to Git and upper management has been touting it as a big success story in emails. The thing is, everyone I speak to in the project hates it and not just because it’s new and they don’t like change. We had a simple workflow going, we changed a file and then committed it directly to trunk, before we went to UAT we created a release branch. Life was simple and things worked really well with SVN, you edited a file, committed and that was that. With Git and Odyssey it’s a whole process though, we have to create a feature branch, then commit changes to the feature branch, then pull master branch changes, merge the master with the feature locally, push the changes, ask someone to approve/review every little change, then merge the feature with the master remotely. I mean what a drag. What are we doing wrong?

41 comments

[ 2.6 ms ] story [ 72.2 ms ] thread
> have to create a feature branch, then commit changes to the feature branch, then pull master branch changes, merge the master with the feature locally, push the changes, ask someone to approve/review every little change, then merge the feature with the master remotely

None of that is a requirement of git, that's the workflow your company has chosen. (although branching and code review is usually a good thing)

It sounds like someone's identified a social problem and tried to fix it purely with technology. I'm talking about code review. Knowing no more than what you've told us, I would guess you need some social changes too.

The problem is complex, but the first place I would consider starting: try supplying a break-glass "I want to merge this without review", but have a zero-tolerance policy for unreviewed breakages. If you break something, and it turns out you did so in an unreviewed PR that you just YOLO'ed in, you are buying cake (or whatever). People need to learn from experience that reviews really are great, and one way to show them this is by making it clear how bad it is when there are no reviews.

Note, too, that now you are working with branches, there are tons of probably-unrealised benefits available to you. Pre-merge automated test suites, for example. Personally, I would flatly refuse to work on any moderately-complex code base that did not have automated branch tests that ran before any merge to master. It would almost certainly be well worth some dev effort setting this up, because it genuinely is a huge benefit to the branching workflow.

I really don't think that having yet another process to skip the code review is the way to make it easier to use. Besides, the tooling won't support opposite workflows so this will have to be hacked in somehow (typical git commit hooks that randomly block commits with no explanation unless they have a magic word in the message).

If their team is small (less than 5 developers), don't make code review a hard blocker, there's never anybody available to code review.

Perhaps the company is growing, and has identified that the lack of code review on commits is creating quality issues.
Agreed that it's worth OP diagnosing the actual source of friction here. My answer was implicitly predicated on the assumption of a largeish team, and I hadn't realised that.
You're not doing anything wrong. That's how git works. However..

You're assuming there was nothing wrong with your SVN workflow because it was straightforward and you never experienced a catastrophic failure. I suspect that had your SVN server died you would have had a very bad time. I also suspect you must have a very solid UAT process and developers who really care about the quality of the code they write, because it sounds like there was very little oversight of the code that was going to production - if it did whatever the feature needed and UAT gave it the nod that was all that mattered, right?

With git there are a few assumptions like "One day all of this is going to go horribly wrong." A hard, hard crash of the repo servers for example. With git you can just fire up a new remote and everyone carries on locally while that's happening. With SVN it's a bit harder (although still not a huge problem) but everyone basically has to down tools while it's fixed. That's a problem on big projects. Also with git the assumption is that the code is important in itself. It's more than just whether it works, but whether or not it's something everyone agrees on. The process of pull requests gives people an insight of how the codebase is changing, and enables more people to have an input. Pulling the base branch in to your local branch and fixing conflicts locally means your base branch should always work. It's always deployable. With SVN it's easy to accidentally break things and block a deploy until it's resolved. That's an advantage that's rarely necessary but an absolute godsend if someone is working on something hard that has potential to wreck havoc across all the features.

git is great but a bit annoying when everything is going right. It's worthwhile embracing it for when things are going wrong.

Sounds like you are not taking the time it takes to get to know the concepts in git or are not given the time. One does not become proficient in a DVCS in a day. It takes time and practice. It is important to note, that git is distributed. It is a big leap from SVN. What you perceived as simple before might have been a serious risk everyone took.

I've once tried to use SVN before I learned about git. I was terrified to do something wrong. I did not understand the concepts well. I just did not get it. Fortunately I did not stick around with it, but learned a little bit of git shortly after. Then simply Used what little I knew. Then at some point tagging. Then feature branches. Submodules. Pull requests. You learn step by step and don't get overwhelmed by git's complexity.

I'm keeping an open mind definitely, maybe it's a matter of time and learning the ins-and-outs as you mentioned. My number one philosophy though has always been KISS (Keep It Simple Stupid) and I think I’m off put by complexities in general, not because they’re difficult to learn or understand but just out of principal I think simple is good and complex is bad. Will give it more time though and see.
beware of the difference between simple and too simple though!
The time is always there, but not letting getting angry about a change eat it up and providing an excuse ("I'll just procrastinate, that'll show them!") is key. That means taking the biggest hurdle to learning. Mngt/Life/You could have not forced the change, but that's how it goes.

In case of git, try this: https://www.youtube.com/watch?v=2sjqTHE0zok

SVN has much simpler concepts and a much smaller "surface area" to learn because it is a centralized versioning system. If you don't need the features of a distributed versioning system then there's little reason for using git (apart from "everybody's using git", which of course may be the strongest reason to adopt it too).
Git branching and merging vastly out classes svn, coupled with its ability to version directory deletes and renames which svn struggled with, it makes much more sense than svn.

Most SVN workflows did not use branching and merging, becuase it was such a pain in the ass with SVN.

You folks are right about the confusion between git and the workflow, most folks use gitflow which uses master, develop and feature branches. Others like gitlabflow are quite different, but still make use of git's far better branching and merging.

Correction: Most company workflows did not use branching and mandatory code review because they started 10+ years ago and that was not widespread practice at the time... for a variety of reasons.

Usability has always been a problem as well as tooling support (merging algorithms really used to suck and generate conflicts all over the place, not specific to SVN or to git). Git made some things easier and some things harder. I'd personally rather have a sane command line interface (that I use every day) than the ability to track a rename twice a year.

(comment deleted)
You didn't just change the source control tool you also fundamentally changed the development process. You used to do continuous integration. Now you are doing feature branching instead.

Feature branching is nice in some cases. For example in open-source project this allows a maintainer to review code (of a potentially unknown developer) before it is accepted. It also prevents a contributor to vanish leaving half-finished features.

I believe feature branching is inferior to continuous integration for most private software development teams. Feature branching along with systematic code review add a lot of friction to the integration of a developer's code into the mainline. The consequence is that developers push less commits that become bigger and integrating each developer's code together become a painful process. This starts a negative feedback loop.

If I were you I would look for the evidences that continuous integration make a team perform better and push to keep git but change the workflow back.

Don't get it. You're trying to say that pushing to master with no code review is "continuous integration"?
Continuous integration is when the developers merge their code into a shared master branch at least once a day, ideally several times a day.

A mandatory code review before merging is allowed (that takes some time to be processed) leads to fewer, bigger merges. Imagine waiting for reviews of the 10 pull requests of wrote two days ago, the 10 of yesterday and 5 you created so far today. It's too unwieldy. The latency encourages you to start writing fewer bigger pull requests more spaced out in time.

As an aside, it also tends to reduce trust and cooperation within a team : reviewers become the gatekeepers of the work of contributors. That work is explicitly not trusted until it is checked. Not a great social dynamic I think.

> As an aside, it also tends to reduce trust and cooperation within a team : reviewers become the gatekeepers of the work of contributors. That work is explicitly not trusted until it is checked. Not a great social dynamic I think.

This seems to presume a split between reviewers and contributors that hasn't existed explicitly anywhere I've worked, and has only implicitly been a thing in a handful of contexts. Usually, reviewers and contributors (... well, authors - I would call good review a contribution!) are the same set of people.

Holy shit the amount of made up things you just wrote. "Systematic code review adds friction"? At Microsoft we switched from Perforce or whatever the old centralized system was to gitfs and productivity tripled. We didn't have to manage dumb zip files anymore. We already had code review because what is wrong eith you why wouldn't you have it. Why does continuous integration contradict feature branching? How do you have stable versions of the software if you have unfinished features and hacking in the master version?? I guess this makes sense for very small teams early in the development process but then you can just skip feature branching in git for a while. Git is a godsend
> Why does continuous integration contradict feature branching?

With feature branching, you wait as long needed before the feature is finished. You can only merge when it is fully finished. This goes directly against the continuous integration philosophy of committing early and often.

> How do you have stable versions of the software if you have unfinished features and hacking in the master version?

It requires a strong commitment to ensure that the current state of the code remains healthy (strong unit tests,...). New in-progress features can remain disabled through feature flags, not showing the corresponding UI or not calling the code.

> I guess this makes sense for very small teams early in the development process but then you can just skip feature branching in git for a while.

It's definitely not for every team.

Martin Fowler presents in detail the 'made up things I just wrote' in this article : https://martinfowler.com/articles/branching-patterns.html . See in particular the Continuous Integration section and the following sections.

What is the point of disabled features in master except to not have to have feature branches. Making sure it compiles? You aren't really using those code paths are you? You should have unit tests on your feature branch anyways
You're doing nothing wrong, if you have a well-working svn-based workflow there's absolutely no reason to switch to git, it definitely won't make things simpler or more efficient in your case.

Git has been built for a very specific decentralized development workflow, for a close-knit team with a central versioning server git is simply not the right versioning tool, since you don't get any benefits from what makes git different from svn.

I came looking for a post like this.

"If it ain't broken don't fix it"

There’s too little information in this post. As other have said your complaint aims at the workflow not the technology and you’ve provided none of the reasons your company adopted the new workflow. I can think different situations to which alternatively each one of which is better suited.

Though doubtlessly this new workflow is harder on you than the previous.

Well to me that sounds more of a process matters than a choice of technology.

I use git day to day and before that used svn for a number of years. Now on the odd codebase, I tend to find svn is still in use, and I don't mind that, I just work with it.

From what you've mentioned, it does not seem that introducing git in your organization was solving a problem per-se, you "could" have the exact same flow you had with svn in git, have a single branch and call it whatever you like, e.g. "master", "develop", whatever...

What I think is happening is that someone didn't like the process you had and tried to solve/replace that with a more modern way, e.g. having feature branch, release branches, master, etc... and that it easier to technically achieve with git than with svn. Imagine doing what you doing now with svn, that'd be a nightmare, wouldn't?

On the other hand, your "new" workflow might be solving problems that you didn't realize you had. I.e. code review and approvals have the great benefit of having a 2nd pair of eyes reviewing your changes as well as spreading the knowledge throughout the team as to what's been changing, more like keeping a pulse... so that is not necessarily a bad thing.

> What I think is happening is that someone didn't like the process you had and tried to solve/replace that with a more modern way, e.g. having feature branch, release branches, master, etc... and that it easier to technically achieve with git than with svn.

Either that, or someone read up on what are often considered best practices with git, and then pushed that as the new process in order to be diligent when git was adopted.

There's not enough context to tell whether the process change is a solution to an actual problem or doing it because others are doing it, too.

Most of the stuff (feature branches, review of major merges) are good ideas to do, though, and something you begin to appreciate once you've got used to them. Honestly, I want a feature branch at approximately the point where I realize the feature change is going to take more than two commits or is nontrivial in almost any way. I'm not sure about requiring a review/approval for every little change if it's earlier in development, and I can understand frustration with that, but that's not really a tool-mandated process, as you say.

It looks like to me you never had to do code reviews before, then management decided to push mandatory code review down your throat while migrating to git. Now you have to learn to do code reviews AND to use git AND to do branching, at once. No wonder it's overwhelming.

Code review is normal software development in all companies. Though if your team is small (somewhere less than 5 developers), it's better to make reviews optional, there's never anybody available to review.

As for git and (git) branching, you will have to learn because that's what the industry standardized on. It's a shame because git objectively sucks (it is terribly user unfriendly). You have my deepest sympathy.

Great question! It rings some bells in me.

You have to sell/appreciate the problems before you sell/appreciate the solutions. Git is a solution for some problems, that seems you have either solved with another solution or have (learned to) tolerated for so long that you don't regard as problems.

I started git with only a single master branch, write code, commit and push. If code breaks, we fix it. The problem is the lack of code review. Without review, we are not even conscious about how much bad code went in, or how much better code could go in. And while we were always happy coding, deep down, we also have a bit unsatisfactory that we don't get to discuss our code, missing opportunities boast our clever idea and deep foresight ...

But once we are convinced that lack of code review is the problem, accepting the git branch review workflow is easy, and we appreciate it ever since.

It rings bells of many other technologies. Some of them I had hard time to sell because others have all used to the problem since the beginning and just don't see the need to fix. Some other technologies folks are just accepting because other people are touting it, applying to their "problems" that does not exist in the first place.

> What are we doing wrong?

Let me answer your question with your own quote:

> The thing is, everyone I speak to in the project hates it and not just because it’s new and they don’t like change.

Tech industry is all about change, and since there is so much change, we have to rely a lot of having good processes that can be applied to all sorts of changes. It does sound like your company dropped a bomb of a bunch of changes - they probably could have baby-stepped into code review before also switching to git. Too much change is hard for anyone. But I don't think your company's problem is git.

I used to be the guy that had a single branch project with pushes directly to master, live changes on production, some days had multiple deploys as we discovered bugs. While that was _easy_ from the perspective of "hey, I could just push code, and it was great", it was hard because all deploys were high-stakes, and could mean spending hours debugging and redeploying into the after hours of my work day (and this happened regularly).

Just to ask a question back to you, how many times did your company's simple SVN process backfire and cause more work for you?

I am unaware of SVN backfiring, however I do see peer reviews as being grossly overlooked in the project. However, Git does not solve that problem because people just click the approve button in Odyssey without actually reviewing the code.
Just like traffic lights and road signs don't prevent accidents because some drivers ignore them.

Surely Odyssey keeps a record of who approved what. If you approve something, your name should be attached as having reviewed. Then you share some of the responsibility if the commit is bad.

Oh, to be clear, I'm not saying that SVN itself backfired, but the process they used, which sounded a lot like firing from the hip for new features.
The steps described sound about right, so I wouldn't say that anything is being done wrong. I'm not aware of Odyssey. Somehow though I would consider the steps easy rather than a big chore. How much of the friction is the actual git operations rebase/merging vs the reviewing? If it's the reviewing step that's a culture thing where everyone has to buy into timely feedback to run smoothly.
> ask someone to approve/review every little change

If you're asking people to review your edit history, that's (IMO) something you're doing wrong. Branch early, commit often, but clean it up into a series of conceptually related, easy to follow changes with good commit messages before you push it anywhere (morally) shared. In SVN, commits are always meant for the world. In git, commits are meant for you until you decide to share them. Owning that helps. (Again, IMO/YMMV/&c...)

With git it's also possible to commit directly to master. An SVN-like workflow is entirely possible. But git can do more, and allows very different workflows that allow a more distributed way of working.

Of course your company needs to use a workflow that works for what you need it to do. It's also possible that the company needs to invest in a bit of training here. Having your employees working with a version control system they're not comfortable with can become a big problem.

I think this is the right response.

OP is complaining about the workflow/process not about the tool. Instead of all that big process described, the team could simply git-clone upstream, work on their local main (formerly master) branches, then "git commit && git push && git pull" all the time and that's it: you got your old workflow back, but now with git.

I think you guys hit the nail on the head here, we want our old workflow back! I'm not saying that the commit to master workflow is the greatest in the world but it worked for us and now we're blaming Git for this feature branch workflow which we admittedly we also tried with SVN but ended reverting back to the simple commit to trunk workflow.
There's a good chance that your employer switched to git because they want to enable feature branches and code review on pull requests.
>We had a simple workflow going, we changed a file and then committed it directly to trunk, before we went to UAT we created a release branch. Life was simple and things worked really well with SVN, you edited a file, committed and that was that.

You can do that with Git.

>With Git and Odyssey it’s a whole process though, we have to create a feature branch, then commit changes to the feature branch, then pull master branch changes, merge the master with the feature locally, push the changes, ask someone to approve/review every little change, then merge the feature with the master remotely.

Based on what you wrote, this seems all about the workflow your company chose to use with git, not git itself.

Your organization has chosen a poor workflow.

Developers should never be required to produce and merge "feature branches" but only properly rebased sequences of commits which keep the history linear.

With a rebase workflow, you can juggle multiple changes in the same branch (your local main branch that tracks the remote master).

I sometimes have as many as five or six separate work items going on. This is much more productive than switching among five or six branches which don't know about each other, not only due to the convenience, but also due to the integration: all six things build together and are being tested in the same image.

Git has a great "interactive rebase" feature (git rebase -i) for rearranging the order of your local commits, and combining them together. Thanks to interactive rebase, I submit properly rebased versions off all six of those items to the review system.

It's also worth learning how to use "git commit --fixup" and its variant "--squash", together with "git rebase -i --autosquash". You can make commits which are amendments to commits that are not the current HEAD. The auto-squash rebase will automatically put these commits in the right order, and configure them to be combined with their target commits. I have the --autosquash option set up implicitly in the global config.

Code review is an excellent process. I've been saved from silly mistakes more times than I can remember, and have saved others. When I'm working on my solo side projects, I'm keenly aware of the lack of review now. I've screwed up many times in ways that a second pair of eyes could have prevented.