It's nice to see LWN on HN for the second time in one day, but please remember: it is only LWN subscribers that make this kind of writing possible. If you are enjoying it, please consider becoming a subscriber yourself — or, even better, getting your employer to subscribe.
Bjarmason has a good response about the practicalities of an attack; it explains why a "broken" hash is rarely a running-around-with-your-hair-on-fire level emergency. It would clearly be better to use a better hash, but is it actually urgent for anyone? Probably not.
Does the usage of SHA-1 in Git actually have security implications, though? It's basically only used to generate addresses for refs and hunks and all that.
I don't think it does; sure, someone could potentially craft a malicious commit that causes a SHA1 collision in your repo, but I think if you are merging commits from malicious authors, you've got way bigger problems than that.
...and if you're merging commits from a developer who, unknown to either of you, had their laptop compromised and their repo corrupted? Remember that the compromise of kernel.org happened via a developer's laptop, and it was only the security of the hash chains that preserved confidence in the repositories stored there.
As noted in the article, an SHA-1 collision attack does not appear practical now, but that is a situation that can change.
GitHub actually makes pull requests available as an unlisted part of the original repository under refs/pull/$PR/head and refs/pull/$PR/merge, which allows a malicious author to add themselves to your index, without your involvement.
Not to say that this attack is in any way practical, yet. Just that some providers don't require active involvement to try and attempt it.
If a repo accepts third-party contributions, you can create a split brain where half the people see one set of contents and the others see a different set, but the same hashes are available.
I don't know if this would survive additional commits on top as I'm not familiar enough with git's internals.
"Given the threat that the SHA-1 hash poses, one might think that there would be a stronger incentive for somebody to support this work. But, as Bjarmason continued, that incentive is not actually all that strong. The project adopted the SHA-1DC variant of SHA-1 for the 2.13 release in 2017, which makes the project more robust against the known SHA-1 collision attacks, so there does not appear to be any sort of imminent threat of this type of attack against Git. Even if creating a collision were feasible for an attacker, Bjarmason pointed out, that is only the first step in the development of a successful attack. Finding a collision of any type is hard; finding one that is still working code, that has the functionality the attacker is after, and that looks reasonable to both humans and compilers is quite a bit harder — if it is possible at all."
There is a big difference between having 2 file with the same garbage comment but different content that have the same hash and creating a new file that had a garbage comment and has the same hash as some other file not chosen by the attacker. (preimage vs collision).
Sha1 has a collision attack. We are far away from a preimage attack
There is a middle course: You could get a pull request accepted with good content, but including a sensible comment whose exact wording you can choose, so later you can replace the contents of that commit with malicious code and a garbage comment. Such a collision is easier to create than a preimage attack (because you have some control over the preimage), but harder than if you could choose the preimage arbitrarily (which wouldn’t be accepted in the pull request). I admit that I have no idea how to quantify the difference in difficulty.
To elaborate a bit: One thing that makes a viable attack against Git especially hard is that aside from the hash it's using has a behavior of never replacing an already hashed object[1].
So let's say I have a tool that can take a given file & SHA-1 pair and produce a collision, the next step is quite hard. I could in this scenario produce a file with an exploit whose hash matches that of Linus's kernel/pid.c or whatever.
But how do I get that object to propagate among forks of linux.git to distribute my exploited code?
If I e.g. push it to a fork of linux.git on a hosting provider that Linus uses the the remote "git-index-pack" process will hash my colliding object, but before it stores it check whether such an object ID exists in its object store, if it does it'll drop it on the floor. You don't need to store data you've already got in a content-addressable filesystem.
Which is not to say that a hash collision is a non-issue, and Git should certainly be migrating from SHA-1. There's no disagreement about that in the Git development community.
But it matters for how much you should panic how the software you're using could be exploited in the case of a hash collision.
Also, the scenario above presupposes a preimage attack, which is a much worse attack on a hash function than a collision attack. Currently no viable preimage attack on SHA-1 exists, only a collision attack.
Which means that before any of the above I'd have to have produced a viable version of say kernel/pid.c that Linus was willing to merge, knowing that my evil twin of that version is something I intended to exploit people with.
Then I'd need to patiently wait for that version to make it into a release, knowing that even a one-byte change to the file would foil my plans...
1. On the topic of running with scissors: I wrote a patch to disable that collision check for an ex employer, it helped in that I/O-bound setup, and we were confident in the lessened security being a non-issue for us in that particular setup. The patch never made it into git's mainline. The patch won't apply anymore, but the embedded docs elaborate on the topic: https://lore.kernel.org/git/20181113201910.11518-1-avarab@gm...
I had to look, but in the case of libgit2 yes they have. Like git they have a way to select SHA-1 backends, and the default is the SHA1DC library.
But, even supposing a libgit2 that didn't use SHA1DC I think most users would be protected in practice if the "git" they use used SHA1DC. Hosting providers, local editors etc. use libgit2 for a lot of things, but I think in most cases (certainly in the case of the popular hosting providers) it's some version of "/usr/bin/git" that's handling your push, and actually propagating your objects.
For stopping a colliding hash it's enough that any part of the chain of propagation is able to stop it.
I think you're confused here (or maybe I am). In this scenario you e.g. would be replacing the blob object for kernel/pid.c.
Whereas the GPG signature facility in git is creating a "tag" object with a signature of the preceding part of the object envelope, and that envelope refers to another object type. E.g. in the case of signed tags usually a commit object.
So if you are able to replace the blob the GPG signature will still be valid, since it relies on signing the SHA-1 DAG.
There is e.g. the third-party git-evtag[1] which gets around this problem by recursively unpacking the signed content, which does give you the advantages of the stronger hash. But this isn't how signed content in Git itself currently works.
I think there was some recent-ish discussion of having something like that in git itself, and perhaps I've missed something, but I'm pretty sure I didn't miss the tag signing code doing a full object walk, which is what you'd need for signing a SHA-1 Git repository in a way that didn't piggy-back on SHA-1's security.
Ah, sorry. I entirely misunderstood & misread what you were saying.
But in that case you'd still have the same collision, as surely what you're producing a collision for is the tree or blob object(s) you're asking to be merged in, as opposed to the hash for the commit object (which will change as you amend it). No?
I think the actual issue here is environment accreditation not allowing the use of sha-1 at all, but that is still rare. It'll become a much larger issue if a future FIPS standard ever disallows sha-1, because that will impact a ton of environments. It means git won't even work on your servers any more.
Example: `git commit -a -S -m 'signed commit'` signs the SHA-1 hash directly.
Even if the SHA-1 digest is rehashed with a secure hashing algorithm, SHA-256, it would hide the fact that the reference is to an insecure hashing algorithm. The project itself needs to be rehashed with a secure hashing algorithm for signing to be secure.
It's more complicated than that. If the most recent signatures are entirely based on SHA-256, and you trust those signatures sufficiently, then they act as protection for all ancestor commits. In that case a SHA1-based signature on an older commit isn't a big deal.
>then they act as protection for all ancestor commits
How does that work? My understanding was that a git gpg signature only signs the project at that commit state.
It says nothing about past (or future) commits outside of a digest reference to past commits, which if that digest wasn't upgraded, would be considered insecure.
Said another way: Git does not rehash past commits, or the present commit, when gpg signing. A commit itself only includes the SHA-1 digest of the previous commit.
If you convert a repo to SHA-256, then surely it will recalculate all the hashes back to the start, right? Otherwise that's not a conversion. And then new signatures will use a hash that's SHA-256 all the way down.
The old signatures will still be SHA-1. But if you try to replace any part of a commit, the SHA-256 won't match. So the combination of "the commit is an ancestor of multiple securely signed commits in this repo" and "the SHA1 on the signature matches" is enough to know you have the right data in most use cases.
You are correct. In the AdES signature world, the solution is to have a cryptographic (signed) timestamp using a newer hash algorithm that rehashes all previous commits, and to include that timestamp into a new commit. When verifying the hashes of old commits, the software would verify that those are covered by an appropriate timestamp that proves that they were created before the old hash algorithm was considered too weak.
This is very similar to the following: Instead of rehashing, i.e. replacing old hashes with new hashes, add the new hashes alongside the old ones, and sign the new hashes, together with the time mark, by a trusted authority. The old hashes and signatures then remain valid indefinitely as long as the new hashes and signatures are verified successfully.
In case anyone has forgotten, the process for pushing it to your own server is three shell commands. You run, on the server:
git init --bare public_html/mything.git
cd public_html/mything.git/hooks/
mv post-update.sample post-update # runs git update-server-info on push
(This assumes that your public_html directory exists and is mapped into webspace, as with the usual configuration of Apache, NCSA httpd, and CERN httpd. If you don't have an account on such a thing you can get such PHP shared hosting accounts with shell access anywhere in the world for a dollar or two a month.)
And then on your dev machine, it's precisely the same as for pushing to Gitlab or whatever, except that you use your own username instead of git@:
git remote add someremotename user@myserver:public_html/mything.git
git push -u someremotename master # assuming you want it to be your upstream
Then anyone can clone from your repo with a command like this:
git clone https://myserver/~user/mything.git
They can also add the URL as a remote for pulls.
If you want them to be able to push, you'll need to give them an account on the same server and either set umasks and group ownerships and permissions appropriately or set a POSIX ACL. Alternatively they can do the same thing on their server and you can pull from it. There are reportedly permission bugs in recent versions of Git (the last five years) that prevent this from being safe with people you don't trust (https://www.spinics.net/lists/git/msg298544.html).
Of course source control is only part of the overall development project workflow, so for many purposes adding SHA-256 support to Gogs or Gitlab or Gitea or sr.ht is probably pretty important: you want a Wiki and CI integration and bug tracking and merge requests. But the git repo still works fine with a bog-standard ssh and HTTP server, though slightly less efficiently. It's easier than setting up a new repo on GitLab etc.
Running a git repack -an && git update-server-info in the repo on the server can help a lot with the efficiency, and for having a browseable tree on the server as well as a clonable repo I put this script at http://canonical.org/~kragen/sw/dev3.git/hooks/post-update:
That's very far from being GitLab (contrast http://canonical.org/~kragen/sw/dev3 with any GitHub tree view), and it's potentially dangerously powerful: if you're doing this in a repo where you pull from other people, and the server is configured to run PHP files or server-side includes in your webspace (mine isn't!) or CGI scripts (mine is!), then just dropping a file in the repo can run programs on the server with your account privileges. This is great if that's what you want, and it's a hell of a lot better than updating your PHP site over FTP, but that code has full authority to, for example, rewrite your Git history.
In theory you can do other things from your post-update hook as well, like rebuild a Jekyll site, send a message on IRC or some other message queueing system, or fire off a CI build in a Docker container. (Some of the...
People also forget about Gitolite, which provides lightweight shared access control around Git+SSH+server-repos. For me it's a much simpler alternative than systems with a heavyweight web UI. Although to be honest I don't know whether Gitolite handles SHA256 hashes (I've never tested it).
I did forget about Gitolite! Thanks for the reminder! Do you have suggestions for what sorts of CI tooling and bug trackers people might want to use with it?
I started a new company a year ago (https://hydraulic.software - launching soon) and we use Gitolite in a somewhat custom stack rather than a forge. At some point I'll blog about it because it's a rather unusual setup but so far it's working quite well.
Hydraulic's first product is a packaging tool for desktop apps and we're mostly a JetBrains shop. The gist is:
- A large dedicated machine in a cheap colo provider (Hetzner), with
- Gitolite with some custom configs
- YouTrack for tickets
- TeamCity master, some agents and a Windows VM for testing.
- Dedicated Mac hardware in the office for Mac CI testing, also running TeamCity agents.
The workflow is a homegrown one that we call "git oriented review". I'll briefly describe it and then discuss why we use it:
1. (Almost) Every git repository is owned by someone specific. There are no shared repositories. All code flows upwards to my repository via merges, kernel style, and that's the one that's used for releases.
2. Gitolite is configured to allow users to push into each others repositories but only under a special branch namespace (rr/$user/whatever). Other branches are protected and yours alone. You also have a personal set of build configs in TeamCity, so if you want to create a customized CI setup you can, and so branches you push to your personal repo don't interfere with the greenness of anyone else's builds.
3. To submit code for review, you push a "review request" branch in the rr/$user namespace of the reviewer's repository. The commit message(s) have a command in them that's interpreted by YouTrack once the build goes green to update the ticket, which in turn then notifies the reviewer that new code exists and is green. The notification can be via email, or IDE notifications, or Slack etc. Lots of options, up to you.
4. The reviewer then makes a code review by adding commits to the rr branch. For small changes, the reviewer just makes the change directly. For larger changes they add a //FIXME comment to the code. FIXMEs may not be merged into non-rr branches, so they are a request for the original author to remove them by e.g. fixing the issue, or adding a comment to explain why in reality it's not meant to be fixed. Thus code and commits are used as a type of discussion forum. You can of course also just jump into a CodeWithMe session to do a bit of pair programming on it for more complex discussions (everyone is currently remote at Hydraulic so everything is done via tools).
5. When satisfied the reviewer merges the rr branch to their own master or dev branches and deletes it. The merge commit contains another command to mark the ticket as fixed, again, it's only applied if the build goes green. At this point they "own" the result because it's in their personal repository. Finger pointing isn't allowed.
I designed this workflow due to lack of satisfaction with the GitHub PR based workflows used at my previous firm, which was a fairly typical centralized one involving a single shared repository, a protected master branch with people pushing ad-hoc branches into it and then opening PRs for reviews. The problems I wanted to fix were:
• Reviewers would only respond with comments because that's what the GitHub workflow promotes. Often many comments were small and it would have been both much faster and also more collaborative for the reviewer to make the change directly, but lack of clear ownership over branches made conflicts likely, and people were reluctant to do this.
• Sometimes what the reviewer wanted wasn't obvious. Again if they could have made the change directly it would have been better.
• Reviewers would often get tired after enough comments, or more than a few rounds of review, especially if the dev wasn't actually applying all the requested fixes. So they'd end up waving code through ...
Interesting! I'm sure you thought of the possibility of having reviewers pull code they are asked to review rather than having it pushed to a non-master branch in their repo, which would have avoided the Gitolite headaches; what are the advantages of the push model?
The push is the "pull request" in this model. It triggers CI, which in turn triggers notifications to the reviewer that there's pending work. The `rr` namespace acts as the inbox for requests. Deleting the `rr` branch is like closing it.
There's also clarity over ownership that way - the `rr` branch is owned by the owner of the repository and they can push to it, to make it into whatever form they want.
Devs may also explicitly send a message indicating there's work to be reviewed, but they don't have to.
It would have made sense to say that in 01990 when the hardware cost US$12000 and the software required constant hand-feeding. But now, virtually every home internet connection has a server built into the cable modem, you can rent a VPS for US$5 a month, and you can bring up a running nginx configuration with a single docker command.
Running a server isn't any more difficult than running an Ubuntu laptop — in fact, it's mostly the same tasks, except that you can version-control the server setup in Git — and considerably more educational.
So I would say that most developers don't run their own server, and that's a criminal failure of education that imperils the future of civilization.
> Adding my own 0.02, what some of us are facing is resistance to adopting git in our or client organizations because of the presence of SHA-1. There are organizations where SHA-1 is blanket banned across the board - regardless of its use. [...] Getting around this blanket ban is a serious amount of work and I have very recently seen customers move to older much less functional (or useful) VCS platforms just because of SHA-1.
Seems like this company could just use the current SHA-256 support then? Especially if it's the type of company that does all its development in-house and there's no need for SHA-1 interoperability.
Except said company likely uses one of the Git forge providers, either in-house or as a SaaS, as the (oxymoronic for git) central repo. Until they support SHA-256, or the company goes with a its own git repo solution that is set up for it, companies won't make the move.
I can name a couple of industries where compliance (and their enforcement arm, security[0]) teams require N+1 different monitoring and enforcement agents on all systems because Compliance[TM]. Due to these agents the systems' IDLE load is approaching 1.00 - on a good day. On a less good you need four cores to have one of them available for workload processing.
0: I use the word "security" only because the teams themselves are named like that. You can probably infer my opinion from the tone.
In a past life I used to work for an anti-virus company who in addition to the Windows product sold the very portable virus-scanning engine for pretty much any other OS you could name. I worked in the *nix department, where we ported it to everything from Linux to the BSDs, HP/UX, Solaris & beyond, as well as more obscure setups like z/OS.
So, we sold people software that would run on some fridge-sized Sun machine running Solaris, to ensure that their Solaris machine wasn't about to get infected with the latest Windows virus.
The occasional support calls with technically minded *nix admins were amusing. We knew that what we were selling them was completely useless and made to secret of that fact, they likewise knew that the software they were running was useless to them. The one thing they cared about was that it didn't contribute to the load, and we did our best.
But some PHB somewhere in their organizations had decreed that all computers everywhere must have an anti-virus scanner, and if you're sufficiently motivated to buy something eventually someone will sell it to you, even while telling you that you don't need it :)
I definitely see your point -- who hasn't seen or heard of companies ruined by officious rulemakers with no clue, rules to make something more secure that do the exact opposite etc. I've seen my share.
But blanket-banning an obsolete and insecure hash algorithm isn't a bad thing, it's entirely reasonable. In this case, as the article makes clear, it's git that's at fault.
> > There are organizations where SHA-1 is blanket banned across the board - regardless of its use.
Reminds me of the time a security audit (which literally just involved running some scanning tool and dumping the results on us) complained that some code I had written was using MD5 - but in a use case in which we weren’t relying on it for any security purposes. I ended up replacing MD5 with CRC-32 - which is even weaker than MD5, but made the security scanning tool mark the issue as remediated. It was easier than trying to argue that it was a false positive.
The big problem with using sha1/md5 in non-secure contexts is:
*Someone later might think its secure and rely on that when extending the system.
*it can make it difficult for security people to audit code later as you have to figure out if each usage is security critical
Using a non crypto hash makes both those concerns go away since everyone knows crc32 is insecure. The alternative of using sha256 also works (performance wise it is close enough, so why not just use the secure one and be done with it.)
I don’t depend on the collision resistance of SHA-1 for the security of my git repos because I don’t accept pushes from people I don’t trust. If I did, objects with hash collisions would not be transferred or (I hope) accepted. Am I missing something?
Granted, signed tags do depend on this collision resistance, but I don’t use that feature. Signing entire releases from a trusted repo seems like a better approach.
It's not just the pushes themselves; anyone who can create commits or blobs that eventually get merged into your repository, directly or indirectly, can potentially engage in a collision attack.
Sure, if you use git with a very closed development model, this doesn't necessarily affect you much. But it's (potentially) a big problem for collaborative open-source projects, because it requires trust in every single contributor. And the trust requirement can't necessarily be mitigated using ordinary means like code reviews.
Collision isn't a spooky action at distance. Even if they tricked the victim into accepting a file they have a collision for, they still can't do anything nefarious. Attack requires an opportunity to replace the colliding file with its evil twin, and that requires write access to victim's repository or tricking the victim into re-fetching their files from an attacker-controlled repository.
Besides, the known collision attack generates files with blocks of binary garbage, which makes it difficult to trick someone into accepting. It won't look like source code, and if someone accepts binary blobs of executable code, you don't need collisions to pwn them.
> Besides, the known collision attack generates files with blocks of binary garbage, which makes it difficult to trick someone into accepting. It won't look like source code, and if someone accepts binary blobs of executable code, you don't need collisions to pwn them.
IDK, I could see this happening in multiple ways.
1. Images / media artifacts stored for display purposes
2. Cached files - 'zero install' config for yarn comes to mind, where every dependency has its file cached in git.
Plus binary files aren't displayed in git diffs so it seems somewhat easy to sneak in.
Otherwise, yeah, agree. Most people don't rely on Git's security model, they rely on Github's.
Upstream Git client says "Binary files a/filename and b/filename differ" whenever it detects changes a binary file. This is mentioned in output of 'git diff', 'git status', 'git show' and other commands.
The article mentions that “none of the Git hosting providers appear to be supporting SHA-256”, but what about self-hosted solutions? In particular, sr.ht. Seems to be nothing[1] in their issue tracker.
Obviously the concern is not the ASICs themselves but the ASIC designers. (Using miners here in the colloquial sense of human collectives/corporations backing the machines than than the specific sense of the raw machines themselves.)
Yes, a practical preimage weakness in SHA-256 is a nightmare scenario with huge implications to the rest of internet security beyond just get. It's why I sometimes can't sleep at night knowing how much energy bitcoin spends daily on a continuous massively distributed partial preimage attack on SHA-256.
> how much energy bitcoin spends daily on a continuous massively distributed partial preimage attack on SHA-256.
I would not be concerned about this. The way the asics operate is they discard the results. Also, the hashes are random strings which don't compress very well, so storing trillions upon trillions of them (for later analysis) is not practical.
Again, the point of the fear is not the specifics of current operations (ASIC details; which y'all are talking about as if all of the miners are using the same hardware), but the fear of future operations and that there's an enormous industrial preimage attack effort at all. One that we can see in real time, in global energy consumption graphs.
Maybe you find "cold comfort" that because we can watch it in real time if someone discovers a weakness we will also watch its repercussions and the subsequent horrifying fall in real time, too, but I certainly don't.
I think the work they're doing to migrate to SHA-256 also includes not hardcoding the hashing algorithm so much into the code, so that it has some forwards compatibility if they need to change again in future.
The canonical solution is SHA-512/256 i.e 512 truncated to 256 bits where "nothing is lost" compared to SHA-256 and something is gained. It might even be faster (due to the 64-bit word formulation of SHA-512) in some implementations.
Generally is faster (fewer rounds per byte). If you have 256 bits available for your hash and you're on a 64 bit architecture, I've yet to see a case where you're not better off for performance and security choosing SHA-512/256 over SHA-256, assuming you have the choice.
Is this still true? I understood SHA256 to be faster than SHA512 due to hardware acceleration on current CPUs; dedicated instructions exist for the former but not the latter.
It seems like it's not true on a MacBook with the Apple M1 processor; SHA-256 is now significantly faster. It still seems to be true on a couple of Intel machines I have access to.
Googling i noticed there seems to be a bug in openssl where it does not use optimized sha-512 on m1 (but does for sha256) - https://github.com/openssl/openssl/issues/14897 so that might be the explanation.
Also i think the length of the input matters when comparing sha256 vs sha512.
Truncated SHA-* hashes are more secure against length-extension attacks, but are very much less secure against collision and pre-image attacks (which are more important in most scenarios).
I don't believe the length is a major issue. It's "upgrading" references to a new hashing algorithm that's the issue.
If for some reason length was an issue, a base64 encoded 256 bit string, like a SHA-256 digest, is 43 characters. That too can be truncated to 40 characters, which has 238 bits of security. SHA-256 is not only a better hashing algorithm than SHA-1 but it could also result in higher effective security even when truncated.
160 bit output, without a cryptographic weakness, is good for about 30 trillion commits per second continuously for 1000 years.
For SHA the cryptographic strength isn't primarily from the length of the hash, but from the internal number of rounds is (e.g. 160-bit SHA-1 with fewer rounds has been badly broken way earlier, and 160-bit SHA-1 with more rounds would be safer).
Cryptographic hashes are designed to be safe to truncate and still have all the safety the truncated length can provide. It's basically a requirement for them being cryptographically strong. Even in the SHA-2 family, the SHA-224 and SHA-384 are just truncated versions of larger hashes.
It makes random collisions more likely when comparing truncated SHA256 to pure SHA256, but given the collisions and pre-image attacks shown so far is truncated SHA256 still safer than SHA1 in that respect? I have seen an article that claimed so (sorry, I can't re-find it ATM so I can't offer it for criticism, if anyone else has good information either way please respond with relevant links), and it is immune to extension attacks which is a significant advantage if this is part of your threat sensitivity surface and SHA1 is used without other protective wrappers like HMAC.
Truncated sha256 is safer than sha-1 (depending of course on how much you truncate it, but given context lets assume truncating to size of sha-1 - 160 bits).
SHA-1 is quite broken at this point. SHA-256 is not. There aren't any practical non-generic attacks on full sha-256 and thus there wouldn't be any on the truncated version. The Wikipedia article goes into the different attacks on the two algorithms.
That said, if your concern is length extension attacks - strongly reccomend using sha-512/256 instead of trying to do your own custom thing.
It's frankly amateurish for the git dev to delay this. The longer this lasts, the more painful it'll be whenever the switch will finally take place.
Linus shouldn't have used SHA-1 in the first place, it was already being deprecated by the time git got its original release. Then every time a new milestone is reached to break SHA-1 we see the same rationalization about how it's not a big deal and it's not a direct threat to git and blablabla.
It'll keep not mattering until it matters and the longer their wait the more churn it'll create. Let's rip the bandaid that's been hanging there for over 15 years now.
> Linus shouldn't have used SHA-1 in the first place, it was already being deprecated by the time git got its original release.
Using SHA-1 to begin with was fine. However, commit hashes should have been prepended with a version byte to make it easier to transition to the next hash algorithm.
This would mean an old Git client could report an error to the user of the nature “please upgrade your software to support cloning from this Git server” instead of failing with an error that’s inseparable from “the Git server is broken” when trying to clone a Git repo using SHA-256.
By the time Git was first released the first attacks on SHA-1 had already been published, but I agree with your general point about allowing for backward compatible updates.
They're talking about the hex representation. It doesn't make sense to think they were referring to the nibble as the version, given that all 16 values of that nibble are already in use.
The problem is not a missing version byte. SHA-256 is trivially distinguishable from SHA-1 by hash length. The problem is that that the length of a SHA-1 hash (20 bytes) is (or was) hardcoded in too many places.
I worked on code signing for civilian aviation years ago and there were people trying to pressure me into supporting MD5 and SHA-1 signatures. I told the first group to jump off a cliff, and the second group got a firm no. The first papers on theoretical SHA-1 attacks had already been published, we were still a couple years out from active use, and people were already beginning to talk about starting to organize the SHA-3 process.
Once a system expects to handle SHA-1, then you have to deal with old assets that have deprecated signatures, and that's a fight I 1) didn't want to have and 2) was fairly sure I wouldn't be around to win.
Git was still brand new, largely unproven at that point, and I don't understand why he picked SHA-1.
Linus' original excuse for using SHA-1 was that Git hash trees and hash identifiers were never meant to be cryptographically secure. GnuPG signing support, the popular belief that Git trees had a strong security property, etc, came afterward, along with increasingly awkward excuse-making.
So strictly speaking Linus and subsequent maintainers weren't being amateurish in the beginning. (You didn't say that explicitly, but it would be a fair criticism given what was known about SHA-1 at the time, including known by Linus--he knew and made a choice.) Rather, in the beginning it was naivety in believing that people wouldn't begin to depend on Git's apparent security properties.
I don't know if/how this played into it, but if you check out the original version of Git whose commit date is April 7th, 2005 it uses OpenSSL for SHA-1.
The first OpenSSL release that has general SHA-256 support seems to have been 0.9.8, released on July 5th, 2005, the code first appeared in OpenSSL's source tree in May of 2004.
Perhaps Linus has commented on it. I don't know, but I wouldn't be surprised if the actual reason is that Git was thrown together as a weekend project, that he vaguely knew SHA-256 was preferable, but his distro's OpenSSL didn't have it yet.
So the initial version used SHA-1 instead, and the rest is history...
Where "h1" is an upgradeable hash (h1 is SHA-256). If there's ever a problem with h1, the hash can be simply upgraded.
Git's documentation describes how to sign a git commit:
$ git commit -a -S -m 'signed commit'
When signing a git commit using the built in gpg function the project is not rehashed with a secure hash function, like SHA-256 or SHA3-256. Instead gpg signs the SHA-1 commit digest directly. It's not signing the result of a secure hash algorithm.
SHA-1 has been considered weak for a long time (about 17 years). Bruce Schneier warned in February 2005 that SHA-1 needed to be replaced. Git development didn't start until April 2005. Before git started development, SHA-1 was identified as needing deprecation.
Just to nit on your portion of signing: wouldn't you need to rehash all prior commits as well so that they used the better hash function? Otherwise someone could find a collision for a prior commit hashed with sha-1, slip that in, and the final commit being hashed with sha256 wouldn't matter.
This then makes the signing code use its own form of hashing that is different from the rest of git's commmit hashing, and seems like a novel way to introduce tooling issues / bugs / etc.
> and the final commit being hashed with sha256 wouldn't matter.
Git stores content, not diffs. So the signature verifies all content stored in that commit. It does t verify anything that came before it, unless those are specifically signed as well.
But the "contents" is just pointers to tree roots with a trusted hash. If the hash is no longer secure, you can't garantee that any such trees are your content, or safe.
The assumption in this context is that all those have been rehashed to SHA-256 too. The point was about whether that rehashing needed to be extended to previous commits.
There's no need to explicitly version your first version of this though. Those first-version values are easy to identify: they don't contain versioning information :)
E.g. say you have `5baa61e4c9b93f3f0682250b6cf8331b7ee68fd8`. What version is that?
Well. It's exactly as long as a SHA1 hash. It doesn't start with "sha256:" or "md5:" or "h1:" or "rot13:". So it's SHA1. Easy and totally unambiguous.
Versioning can almost always begin with version 2.
Me, sowing: "Each record begins with a 4 octet BE value indicating the record length."
Me, reaping: "Each record begins with a single byte indicating the record format version. In version 0, this is followed by a 3 octet BE value indicating the record length."
if you're storing the raw binary rather than hex or base64: yeah. there are often no illegal values, so there's no way to safely extend it, unless you can differentiate on length.
for those, you have to leave versioning room up-front. even 1 bit is enough, since a `1` can imply "following data describes the version", if a bit wastefully in the long run.
That's not applicable to Grox's example. The initial version uses only hexadecimal digits for the SHA256.
If you had: "each record begins with an 8 character record length, in hexadecimal, giving 32 bits", you have no problems. The new version has a 'V' character in byte 0, which is rejected as invalid by the old implementation.
I beg you: please clone git, do the changes, and benchmark them. I bet you won't be able to obtain a statistically significant result from this single branch.
I think the implications for Go are a bit different, though. It's a very simple matter to change the hash algorithm used for go.mod. Even if there was no hash version prefix, it's trivial to add one after the fact, though older tools would probably give a confusing error message without foreknowledge of the concept of an unrecognized hash algorithm. And adding a new hash algorithm is just a matter of writing a relatively small amount of code, and then probably waiting a few Go releases before making it the default and assuming most people will have it.
Git's entire foundation relies on SHA1 hashes. Each commit is its own hash, and contains a list of the hashes of all files that are a part of it. Branches have hashes, tags have hashes. Everything has a hash. A repository that uses a different hash algorithm is a completely different repository, even if the contents and commits are otherwise identical. You can't even store your code on someone else's server (well, aside from manually copying the repository data over, though that won't be too useful) unless that server has upgraded their git version.
I think the argument that gp is trying to make is that it's really hard for git to implement this in a backwards compatible way. You may be right (I don't know anything about Fossil, will take a look!) that Fossil allowed for this by making good design decisions in the past. This is not something that git maintainers can do right now without a time machine though. Old versions are in use out there and will need to keep working if the goal is to make the transition easier for users.
It's a more robust, well-specified, interoperable version of this concept.
Though it's probably overkill if you control both the consumer and producer side (i.e. don't need the interoperability) and are just looking to make hash upgrades smoother, in that case a simple version prefix like Go's approach described above has lower overhead.
> Instead gpg signs the SHA-1 commit digest directly
A minor correction: when signing a commit, gpg does not sign the SHA-1 digest of that commit. This is impossible since the signature becomes part of the commit header which is one of the inputs to the hash function that produces the oid.
Instead, GPG signs the serialized data (parents,headers,tree,message) which would otherwise be the input to SHA-1. Then the sig is inserted into the buffer at the end of the header and the string is digested to produce an oid.
Perhaps you misinterpreted my reply. I didn’t intend to dispute parent’s claim, merely to correct a mechanical detail.
Git signatures are more or less only as secure as SHA-1, although the properties it relies on are not yet compromised and several factors mitigate the real-world risk.
A practical preimage attack on SHA-1 would seriously undermine the security of git signatures since the string being signed includes two or more SHA-1 hashes representing a content snapshot and the commit ancestry. Arbitrary preimage attacks would make it possible to modify a repo’s contents or history without invalidating oids or signatures.
In practice only collision attacks have been found, all of which have a detectable signature that git has been modified to detect.
Disclosure: I work at GitHub, but am speaking for myself.
libgit2 isn't an official library, and even if it did support sha256 dependents would still need to update, so I really don't perceive this as a missing piece.
If everyone started using sha256 then all these problems would be addressed practically overnight.
Is there an explanation of what would go wrong with the naive approach? E.g.:
- Change the binary file format in repos to support arbitrary hash algorithms, in a way which unambigously makes old software fail.
- Increment the Git major version number to 3.0
- Make the new version support both the old version repos and the new ones. Make it a per-repo config item that allows/disallows old/new hash formats. In theory, there's nothing wrong with having objects hashed with mixed algorithms as long as the software knows how to deal with that.
- The old format will probably have to be supported forever because of Linux.
Most user-facing utilities don't care what the hash algo actually is, they just use the hash as an opaque string.
Releasing new software is the simple part. The problem is that versioning is lacking in the old software, and therefore it doesn’t know how to talk to the new software. So for the old software there’s no difference between “invalid data” and “I’m too old, please upgrade me”.
> So for the old software there’s no difference between “invalid data” and “I’m too old, please upgrade me”.
And why is this an issue? Release the new version that can read new repo formats, but doesn't write them yet. Wait a year. Release new version that can write new repo formats and encourage users to upgrade.
Anyone who hasn't upgraded in the past year probably doesn't care about security and should be left behind. Besides, once they google the error message they'll figure it out soon enough. It's not like git is known for its great UX anyway.
I might be mistaken, but github could be using their own version of git and accompanying tools. So, unless they implement uprade themselves, no amount of waiting will make git interoperable with them.
> In theory, there's nothing wrong with having objects hashed with mixed algorithms as long as the software knows how to deal with that.
That's an interesting idea, actually. I'm not sure they plan to support that, though? That would make things a lot easier on existing repositories; without support for mixed hashes, repos would have to have their history entirely rewritten, which would invalidate things like signed commits/tags.
there is one hash version, plus a translation table for the other format. no history rewrite.
new repos will use the new hash.
old repos will eventually fully convert to the new hash, then all old hash links after the transition period will become obsolete.
Anyone aware of any exploits tied the SHA-1 weakness in the wild?
(I have seen proofs of concept [1], but never actually heard of an exploit in the wild using it; for example, on: digital certificate signatures, email PGP/GPG signatures, software vendor signatures, software updates, ISO checksums, backup systems, deduplication systems, Git, etc.)
Most security critical systems have switched to sha256 at this point, and making a fresh collision still costs tens of thousands, so people arent really doing it for kicks (that said, once you have one collision you can reuse it for free as long as you keep the same prefix, so the proof of concept can be repurposed with certain constraints).
The most in the wild one i have ever heard of was when webkit accidentally broke their svn repo by checking in a collision.
However you can look at the history of md5 which had a similar flaw which was exploited by the flame malware.
I love the performance of blake3 but my understanding is it's still a bit of the new kid on the block. Blake2 is a SHA-3 finalist so should be perfectly sufficient, plus it has variable digest sizes, reasonably fast, and other nice features.
Either way, anything relying on hashes for data integrity should at least be flexible to the option of multiple hash algos. But with git, it's going to be hard enough as is to change to SHA-256, and I don't know how parametric it'll be.
I'm not aware of any reason SHA-3 would be less secure than BLAKE3. Quite the opposite: SHA-3 has a very large security margin, so large that the authors cut the number of rounds in half for KangarooTwelve.
I give -3 flying ducks about this, and don't want the Git storage format to be diddled with in any way. Git in 2122 should read and write a git repo made in 2010.
Git is not a public crypto system.
If you think a commit is important and needs to be signed, you need to sign the files and add the signature to the commit.
The part of software engineering they don't teach in college is migration. Some of the most creative work you'll do is figuring out how to get from X to Y without bringing everything crashing down around you (or at least only a couple things crashing down at a time).
"Fossil started out using 160-bit SHA-1 hashes to identify check-ins, just as in Git. That changed in early 2017 when news of the SHAttered attack broke, demonstrating that SHA-1 collisions were now practical to create. Two weeks later, the creator of Fossil delivered a new release allowing a clean migration to 256-bit SHA-3 with full backwards compatibility to old SHA-1 based repositories. [...] Meanwhile, the Git community took until August 2018 to publish their first plan for solving the same problem by moving to SHA-256, a variant of the older SHA-2 algorithm. As of this writing in February 2020, that plan hasn't been implemented, as far as this author is aware, but there is now a competing SHA-256 based plan which requires complete repository conversion from SHA-1 to SHA-256, breaking all public hashes in the repo."
This raises an interesting point: given that git has been dragging its feet for so long on the transition to SHA-256, it's better if they were to drag their feet a bit longer and move directly to SHA3-256 too, and never let the current SHA-256 implementation get widely deployed.
GitHub won’t feel any heat about this until Microsoft salespeople start demanding it.
I’ve added to my todo list a reminder to raise this issue with mine. In fact, I’m going to give them a deadline for when we will start evaluating competitors that do support SHA256.
I suspect that most people on HN do not interact with their MS account team. That relationship is probably managed by your CIO or IT department. They probably have monthly or quarterly “business review” meetings. You should get this issue on the agenda of that meeting.
They don't accept pushes of repositories in that format.
The article says "none of the Git hosting providers appear to be supporting SHA-256", and while GH is not mentioned by name (and I applaud them for indeed not strengthening this "git == github-the-brand" trap), I can't imagine GH was left out of scope when checking the major hosting providers.
as the article says, you can create a local git repository with SHA-256 hashes today, and it should work fine...but the moment you try to push your repo up to Github, you'll hit a brick wall.
Gitlab also appears to be lacking support [0], and the same with Gitea [1].
so it's a grey area where Git itself supports SHA-256-based repos, but without the major Git hosting services also supporting them, the support in core Git is somewhat useless.
I also found that confusing in the article. I actually listened to it using text-to-speech and thought it was some brand, like sourceforge. But now I think they just mean any git hosting service.
Just the other day, I was actually forced to downgrade the file hash used in the product I'm working on to sha1 in order to interact with GitHub's APIs efficiently (to avoid having to download the entire file just to recompute a sha256 for matching).
Luckily I've versioned the internal hash so the upgrade path back to sha256 should be as smooth as the downgrade was. I'm still bitter about it though.
> Even if creating a collision were feasible for an attacker, Bjarmason pointed out, that is only the first step in the development of a successful attack. Finding a collision of any type is hard; finding one that is still working code, that has the functionality the attacker is after, and that looks reasonable to both humans and compilers is quite a bit harder — if it is possible at all.
220 comments
[ 2.3 ms ] story [ 285 ms ] threadhttps://lwn.net/subscribe/Info
And the Wikipedia page for LWN, if you’re not familiar with it:
https://en.m.wikipedia.org/wiki/LWN.net
As noted in the article, an SHA-1 collision attack does not appear practical now, but that is a situation that can change.
Not to say that this attack is in any way practical, yet. Just that some providers don't require active involvement to try and attempt it.
I don't know if this would survive additional commits on top as I'm not familiar enough with git's internals.
"Given the threat that the SHA-1 hash poses, one might think that there would be a stronger incentive for somebody to support this work. But, as Bjarmason continued, that incentive is not actually all that strong. The project adopted the SHA-1DC variant of SHA-1 for the 2.13 release in 2017, which makes the project more robust against the known SHA-1 collision attacks, so there does not appear to be any sort of imminent threat of this type of attack against Git. Even if creating a collision were feasible for an attacker, Bjarmason pointed out, that is only the first step in the development of a successful attack. Finding a collision of any type is hard; finding one that is still working code, that has the functionality the attacker is after, and that looks reasonable to both humans and compilers is quite a bit harder — if it is possible at all."
But you could also hide it as a fake lookup table or inline XPM or something like that.
This seems true yet there are no demos or documented attacks using this method.
I think practically speaking it’s kind of a pain to do.
Sha1 has a collision attack. We are far away from a preimage attack
To elaborate a bit: One thing that makes a viable attack against Git especially hard is that aside from the hash it's using has a behavior of never replacing an already hashed object[1].
So let's say I have a tool that can take a given file & SHA-1 pair and produce a collision, the next step is quite hard. I could in this scenario produce a file with an exploit whose hash matches that of Linus's kernel/pid.c or whatever.
But how do I get that object to propagate among forks of linux.git to distribute my exploited code?
If I e.g. push it to a fork of linux.git on a hosting provider that Linus uses the the remote "git-index-pack" process will hash my colliding object, but before it stores it check whether such an object ID exists in its object store, if it does it'll drop it on the floor. You don't need to store data you've already got in a content-addressable filesystem.
Which is not to say that a hash collision is a non-issue, and Git should certainly be migrating from SHA-1. There's no disagreement about that in the Git development community.
But it matters for how much you should panic how the software you're using could be exploited in the case of a hash collision.
Also, the scenario above presupposes a preimage attack, which is a much worse attack on a hash function than a collision attack. Currently no viable preimage attack on SHA-1 exists, only a collision attack.
Which means that before any of the above I'd have to have produced a viable version of say kernel/pid.c that Linus was willing to merge, knowing that my evil twin of that version is something I intended to exploit people with.
Then I'd need to patiently wait for that version to make it into a release, knowing that even a one-byte change to the file would foil my plans...
1. On the topic of running with scissors: I wrote a patch to disable that collision check for an ex employer, it helped in that I/O-bound setup, and we were confident in the lessened security being a non-issue for us in that particular setup. The patch never made it into git's mainline. The patch won't apply anymore, but the embedded docs elaborate on the topic: https://lore.kernel.org/git/20181113201910.11518-1-avarab@gm...
But, even supposing a libgit2 that didn't use SHA1DC I think most users would be protected in practice if the "git" they use used SHA1DC. Hosting providers, local editors etc. use libgit2 for a lot of things, but I think in most cases (certainly in the case of the popular hosting providers) it's some version of "/usr/bin/git" that's handling your push, and actually propagating your objects.
For stopping a colliding hash it's enough that any part of the chain of propagation is able to stop it.
Whereas the GPG signature facility in git is creating a "tag" object with a signature of the preceding part of the object envelope, and that envelope refers to another object type. E.g. in the case of signed tags usually a commit object.
So if you are able to replace the blob the GPG signature will still be valid, since it relies on signing the SHA-1 DAG.
There is e.g. the third-party git-evtag[1] which gets around this problem by recursively unpacking the signed content, which does give you the advantages of the stronger hash. But this isn't how signed content in Git itself currently works.
I think there was some recent-ish discussion of having something like that in git itself, and perhaps I've missed something, but I'm pretty sure I didn't miss the tag signing code doing a full object walk, which is what you'd need for signing a SHA-1 Git repository in a way that didn't piggy-back on SHA-1's security.
But in that case you'd still have the same collision, as surely what you're producing a collision for is the tree or blob object(s) you're asking to be merged in, as opposed to the hash for the commit object (which will change as you amend it). No?
I think the actual issue here is environment accreditation not allowing the use of sha-1 at all, but that is still rare. It'll become a much larger issue if a future FIPS standard ever disallows sha-1, because that will impact a ton of environments. It means git won't even work on your servers any more.
Example: `git commit -a -S -m 'signed commit'` signs the SHA-1 hash directly.
Even if the SHA-1 digest is rehashed with a secure hashing algorithm, SHA-256, it would hide the fact that the reference is to an insecure hashing algorithm. The project itself needs to be rehashed with a secure hashing algorithm for signing to be secure.
How does that work? My understanding was that a git gpg signature only signs the project at that commit state.
It says nothing about past (or future) commits outside of a digest reference to past commits, which if that digest wasn't upgraded, would be considered insecure.
Said another way: Git does not rehash past commits, or the present commit, when gpg signing. A commit itself only includes the SHA-1 digest of the previous commit.
The old signatures will still be SHA-1. But if you try to replace any part of a commit, the SHA-256 won't match. So the combination of "the commit is an ancestor of multiple securely signed commits in this repo" and "the SHA1 on the signature matches" is enough to know you have the right data in most use cases.
This is very similar to the following: Instead of rehashing, i.e. replacing old hashes with new hashes, add the new hashes alongside the old ones, and sign the new hashes, together with the time mark, by a trusted authority. The old hashes and signatures then remain valid indefinitely as long as the new hashes and signatures are verified successfully.
Okay, but that's a pretty big reason! A git repo that can't be pushed to github/lab is... not always useless, but certainly extremely impaired.
And then on your dev machine, it's precisely the same as for pushing to Gitlab or whatever, except that you use your own username instead of git@:
Then anyone can clone from your repo with a command like this: They can also add the URL as a remote for pulls.If you want them to be able to push, you'll need to give them an account on the same server and either set umasks and group ownerships and permissions appropriately or set a POSIX ACL. Alternatively they can do the same thing on their server and you can pull from it. There are reportedly permission bugs in recent versions of Git (the last five years) that prevent this from being safe with people you don't trust (https://www.spinics.net/lists/git/msg298544.html).
Of course source control is only part of the overall development project workflow, so for many purposes adding SHA-256 support to Gogs or Gitlab or Gitea or sr.ht is probably pretty important: you want a Wiki and CI integration and bug tracking and merge requests. But the git repo still works fine with a bog-standard ssh and HTTP server, though slightly less efficiently. It's easier than setting up a new repo on GitLab etc.
Running a git repack -an && git update-server-info in the repo on the server can help a lot with the efficiency, and for having a browseable tree on the server as well as a clonable repo I put this script at http://canonical.org/~kragen/sw/dev3.git/hooks/post-update:
That's very far from being GitLab (contrast http://canonical.org/~kragen/sw/dev3 with any GitHub tree view), and it's potentially dangerously powerful: if you're doing this in a repo where you pull from other people, and the server is configured to run PHP files or server-side includes in your webspace (mine isn't!) or CGI scripts (mine is!), then just dropping a file in the repo can run programs on the server with your account privileges. This is great if that's what you want, and it's a hell of a lot better than updating your PHP site over FTP, but that code has full authority to, for example, rewrite your Git history.In theory you can do other things from your post-update hook as well, like rebuild a Jekyll site, send a message on IRC or some other message queueing system, or fire off a CI build in a Docker container. (Some of the...
https://gitolite.com
https://github.com/sitaramc/gitolite
Hydraulic's first product is a packaging tool for desktop apps and we're mostly a JetBrains shop. The gist is:
- A large dedicated machine in a cheap colo provider (Hetzner), with
- Gitolite with some custom configs
- YouTrack for tickets
- TeamCity master, some agents and a Windows VM for testing.
- Dedicated Mac hardware in the office for Mac CI testing, also running TeamCity agents.
The workflow is a homegrown one that we call "git oriented review". I'll briefly describe it and then discuss why we use it:
1. (Almost) Every git repository is owned by someone specific. There are no shared repositories. All code flows upwards to my repository via merges, kernel style, and that's the one that's used for releases.
2. Gitolite is configured to allow users to push into each others repositories but only under a special branch namespace (rr/$user/whatever). Other branches are protected and yours alone. You also have a personal set of build configs in TeamCity, so if you want to create a customized CI setup you can, and so branches you push to your personal repo don't interfere with the greenness of anyone else's builds.
3. To submit code for review, you push a "review request" branch in the rr/$user namespace of the reviewer's repository. The commit message(s) have a command in them that's interpreted by YouTrack once the build goes green to update the ticket, which in turn then notifies the reviewer that new code exists and is green. The notification can be via email, or IDE notifications, or Slack etc. Lots of options, up to you.
4. The reviewer then makes a code review by adding commits to the rr branch. For small changes, the reviewer just makes the change directly. For larger changes they add a //FIXME comment to the code. FIXMEs may not be merged into non-rr branches, so they are a request for the original author to remove them by e.g. fixing the issue, or adding a comment to explain why in reality it's not meant to be fixed. Thus code and commits are used as a type of discussion forum. You can of course also just jump into a CodeWithMe session to do a bit of pair programming on it for more complex discussions (everyone is currently remote at Hydraulic so everything is done via tools).
5. When satisfied the reviewer merges the rr branch to their own master or dev branches and deletes it. The merge commit contains another command to mark the ticket as fixed, again, it's only applied if the build goes green. At this point they "own" the result because it's in their personal repository. Finger pointing isn't allowed.
I designed this workflow due to lack of satisfaction with the GitHub PR based workflows used at my previous firm, which was a fairly typical centralized one involving a single shared repository, a protected master branch with people pushing ad-hoc branches into it and then opening PRs for reviews. The problems I wanted to fix were:
• Reviewers would only respond with comments because that's what the GitHub workflow promotes. Often many comments were small and it would have been both much faster and also more collaborative for the reviewer to make the change directly, but lack of clear ownership over branches made conflicts likely, and people were reluctant to do this.
• Sometimes what the reviewer wanted wasn't obvious. Again if they could have made the change directly it would have been better.
• Reviewers would often get tired after enough comments, or more than a few rounds of review, especially if the dev wasn't actually applying all the requested fixes. So they'd end up waving code through ...
There's also clarity over ownership that way - the `rr` branch is owned by the owner of the repository and they can push to it, to make it into whatever form they want.
Devs may also explicitly send a message indicating there's work to be reviewed, but they don't have to.
It would have made sense to say that in 01990 when the hardware cost US$12000 and the software required constant hand-feeding. But now, virtually every home internet connection has a server built into the cable modem, you can rent a VPS for US$5 a month, and you can bring up a running nginx configuration with a single docker command.
Running a server isn't any more difficult than running an Ubuntu laptop — in fact, it's mostly the same tasks, except that you can version-control the server setup in Git — and considerably more educational.
So I would say that most developers don't run their own server, and that's a criminal failure of education that imperils the future of civilization.
Seems like this company could just use the current SHA-256 support then? Especially if it's the type of company that does all its development in-house and there's no need for SHA-1 interoperability.
> I have very recently seen customers move to older much less functional (or useful) VCS platforms just because of SHA-1.
A company this dysfunctional has problems far beyond their choice of revision control system.
0: I use the word "security" only because the teams themselves are named like that. You can probably infer my opinion from the tone.
So, we sold people software that would run on some fridge-sized Sun machine running Solaris, to ensure that their Solaris machine wasn't about to get infected with the latest Windows virus.
The occasional support calls with technically minded *nix admins were amusing. We knew that what we were selling them was completely useless and made to secret of that fact, they likewise knew that the software they were running was useless to them. The one thing they cared about was that it didn't contribute to the load, and we did our best.
But some PHB somewhere in their organizations had decreed that all computers everywhere must have an anti-virus scanner, and if you're sufficiently motivated to buy something eventually someone will sell it to you, even while telling you that you don't need it :)
But blanket-banning an obsolete and insecure hash algorithm isn't a bad thing, it's entirely reasonable. In this case, as the article makes clear, it's git that's at fault.
Reminds me of the time a security audit (which literally just involved running some scanning tool and dumping the results on us) complained that some code I had written was using MD5 - but in a use case in which we weren’t relying on it for any security purposes. I ended up replacing MD5 with CRC-32 - which is even weaker than MD5, but made the security scanning tool mark the issue as remediated. It was easier than trying to argue that it was a false positive.
The big problem with using sha1/md5 in non-secure contexts is:
*Someone later might think its secure and rely on that when extending the system.
*it can make it difficult for security people to audit code later as you have to figure out if each usage is security critical
Using a non crypto hash makes both those concerns go away since everyone knows crc32 is insecure. The alternative of using sha256 also works (performance wise it is close enough, so why not just use the secure one and be done with it.)
Granted, signed tags do depend on this collision resistance, but I don’t use that feature. Signing entire releases from a trusted repo seems like a better approach.
Sure, if you use git with a very closed development model, this doesn't necessarily affect you much. But it's (potentially) a big problem for collaborative open-source projects, because it requires trust in every single contributor. And the trust requirement can't necessarily be mitigated using ordinary means like code reviews.
Besides, the known collision attack generates files with blocks of binary garbage, which makes it difficult to trick someone into accepting. It won't look like source code, and if someone accepts binary blobs of executable code, you don't need collisions to pwn them.
IDK, I could see this happening in multiple ways.
1. Images / media artifacts stored for display purposes
2. Cached files - 'zero install' config for yarn comes to mind, where every dependency has its file cached in git.
Plus binary files aren't displayed in git diffs so it seems somewhat easy to sneak in.
Otherwise, yeah, agree. Most people don't rely on Git's security model, they rely on Github's.
They are (albeit not as prominently as they should). And you can add your own diff engines to show full diffs for different binary formats.
Upstream Git client says "Binary files a/filename and b/filename differ" whenever it detects changes a binary file. This is mentioned in output of 'git diff', 'git status', 'git show' and other commands.
[1]: https://todo.sr.ht/~sircmpwn/git.sr.ht?search=sha-256
If a practical pre-image attack on SHA-256 comes around we have bigger problems than git.
Yes, a practical preimage weakness in SHA-256 is a nightmare scenario with huge implications to the rest of internet security beyond just get. It's why I sometimes can't sleep at night knowing how much energy bitcoin spends daily on a continuous massively distributed partial preimage attack on SHA-256.
I would not be concerned about this. The way the asics operate is they discard the results. Also, the hashes are random strings which don't compress very well, so storing trillions upon trillions of them (for later analysis) is not practical.
Maybe you find "cold comfort" that because we can watch it in real time if someone discovers a weakness we will also watch its repercussions and the subsequent horrifying fall in real time, too, but I certainly don't.
It seems like that could ease much of the migration problems if it's not a problem?
Generally though length ext attacks have a solution - HMAC, which is much more secure than truncate.
The more you truncate, the more vulnerable you are to birthday attacks (practically speaking you would have to truncate quite a lot)
Also i think the length of the input matters when comparing sha256 vs sha512.
There's a point where truncating starts to make it weaker, but when you first start chopping off bytes the benefits outweigh the drawbacks.
If for some reason length was an issue, a base64 encoded 256 bit string, like a SHA-256 digest, is 43 characters. That too can be truncated to 40 characters, which has 238 bits of security. SHA-256 is not only a better hashing algorithm than SHA-1 but it could also result in higher effective security even when truncated.
160 bit output, without a cryptographic weakness, is good for about 30 trillion commits per second continuously for 1000 years.
For SHA the cryptographic strength isn't primarily from the length of the hash, but from the internal number of rounds is (e.g. 160-bit SHA-1 with fewer rounds has been badly broken way earlier, and 160-bit SHA-1 with more rounds would be safer).
Cryptographic hashes are designed to be safe to truncate and still have all the safety the truncated length can provide. It's basically a requirement for them being cryptographically strong. Even in the SHA-2 family, the SHA-224 and SHA-384 are just truncated versions of larger hashes.
SHA-1 is quite broken at this point. SHA-256 is not. There aren't any practical non-generic attacks on full sha-256 and thus there wouldn't be any on the truncated version. The Wikipedia article goes into the different attacks on the two algorithms.
That said, if your concern is length extension attacks - strongly reccomend using sha-512/256 instead of trying to do your own custom thing.
Linus shouldn't have used SHA-1 in the first place, it was already being deprecated by the time git got its original release. Then every time a new milestone is reached to break SHA-1 we see the same rationalization about how it's not a big deal and it's not a direct threat to git and blablabla.
It'll keep not mattering until it matters and the longer their wait the more churn it'll create. Let's rip the bandaid that's been hanging there for over 15 years now.
Using SHA-1 to begin with was fine. However, commit hashes should have been prepended with a version byte to make it easier to transition to the next hash algorithm.
This would mean an old Git client could report an error to the user of the nature “please upgrade your software to support cloning from this Git server” instead of failing with an error that’s inseparable from “the Git server is broken” when trying to clone a Git repo using SHA-256.
Once a system expects to handle SHA-1, then you have to deal with old assets that have deprecated signatures, and that's a fight I 1) didn't want to have and 2) was fairly sure I wouldn't be around to win.
Git was still brand new, largely unproven at that point, and I don't understand why he picked SHA-1.
So strictly speaking Linus and subsequent maintainers weren't being amateurish in the beginning. (You didn't say that explicitly, but it would be a fair criticism given what was known about SHA-1 at the time, including known by Linus--he knew and made a choice.) Rather, in the beginning it was naivety in believing that people wouldn't begin to depend on Git's apparent security properties.
Honestly, I think it's fair to say that hashes isn't meant to be a security feature.
But signed tags/commits/etc. probably need a better hash.
The first OpenSSL release that has general SHA-256 support seems to have been 0.9.8, released on July 5th, 2005, the code first appeared in OpenSSL's source tree in May of 2004.
Perhaps Linus has commented on it. I don't know, but I wouldn't be surprised if the actual reason is that Git was thrown together as a weekend project, that he vaguely knew SHA-256 was preferable, but his distro's OpenSSL didn't have it yet.
So the initial version used SHA-1 instead, and the rest is history...
1. https://marc.info/?l=openssl-users&m=135355590501495
example.com/example v0.0.0-20171218180944-5ea4d0ddac55 h1:jbGlDKdzAZ92NzK65hUP98ri0/r50vVVvmZsFP/nIqo=
Where "h1" is an upgradeable hash (h1 is SHA-256). If there's ever a problem with h1, the hash can be simply upgraded.
Git's documentation describes how to sign a git commit:
$ git commit -a -S -m 'signed commit'
When signing a git commit using the built in gpg function the project is not rehashed with a secure hash function, like SHA-256 or SHA3-256. Instead gpg signs the SHA-1 commit digest directly. It's not signing the result of a secure hash algorithm.
SHA-1 has been considered weak for a long time (about 17 years). Bruce Schneier warned in February 2005 that SHA-1 needed to be replaced. Git development didn't start until April 2005. Before git started development, SHA-1 was identified as needing deprecation.
This then makes the signing code use its own form of hashing that is different from the rest of git's commmit hashing, and seems like a novel way to introduce tooling issues / bugs / etc.
Git stores content, not diffs. So the signature verifies all content stored in that commit. It does t verify anything that came before it, unless those are specifically signed as well.
But the "contents" is just pointers to tree roots with a trusted hash. If the hash is no longer secure, you can't garantee that any such trees are your content, or safe.
E.g. say you have `5baa61e4c9b93f3f0682250b6cf8331b7ee68fd8`. What version is that?
Well. It's exactly as long as a SHA1 hash. It doesn't start with "sha256:" or "md5:" or "h1:" or "rot13:". So it's SHA1. Easy and totally unambiguous.
Versioning can almost always begin with version 2.
Me, reaping: "Each record begins with a single byte indicating the record format version. In version 0, this is followed by a 3 octet BE value indicating the record length."
for those, you have to leave versioning room up-front. even 1 bit is enough, since a `1` can imply "following data describes the version", if a bit wastefully in the long run.
If you had: "each record begins with an 8 character record length, in hexadecimal, giving 32 bits", you have no problems. The new version has a 'V' character in byte 0, which is rejected as invalid by the old implementation.
inconsistencies in how data is presented (optional version number) is a pain to deal with in code
Git's entire foundation relies on SHA1 hashes. Each commit is its own hash, and contains a list of the hashes of all files that are a part of it. Branches have hashes, tags have hashes. Everything has a hash. A repository that uses a different hash algorithm is a completely different repository, even if the contents and commits are otherwise identical. You can't even store your code on someone else's server (well, aside from manually copying the repository data over, though that won't be too useful) unless that server has upgraded their git version.
Well, Fossil's database is much better designed, you reply.
That it is!
The intent behind it is obsolescence and phasing out, resulting in an endless make-work treadmill for the users.
If there is ever a "problem with h1", and you neglect to upgrade your data right there and then, five to ten years, it will be unreadable.
stepping through required versions a common operation
It's a more robust, well-specified, interoperable version of this concept.
Though it's probably overkill if you control both the consumer and producer side (i.e. don't need the interoperability) and are just looking to make hash upgrades smoother, in that case a simple version prefix like Go's approach described above has lower overhead.
A minor correction: when signing a commit, gpg does not sign the SHA-1 digest of that commit. This is impossible since the signature becomes part of the commit header which is one of the inputs to the hash function that produces the oid.
Instead, GPG signs the serialized data (parents,headers,tree,message) which would otherwise be the input to SHA-1. Then the sig is inserted into the buffer at the end of the header and the string is digested to produce an oid.
Source: https://github.com/git/git/blob/39c15e485575089eb77c769f6da0...
Git signatures are more or less only as secure as SHA-1, although the properties it relies on are not yet compromised and several factors mitigate the real-world risk.
A practical preimage attack on SHA-1 would seriously undermine the security of git signatures since the string being signed includes two or more SHA-1 hashes representing a content snapshot and the commit ancestry. Arbitrary preimage attacks would make it possible to modify a repo’s contents or history without invalidating oids or signatures.
In practice only collision attacks have been found, all of which have a detectable signature that git has been modified to detect.
Disclosure: I work at GitHub, but am speaking for myself.
If that was all that was left, we could at least be using sha256 for new repositories.
It seems to me the big missing piece is support in libgit2, which is at least showing signs of progress:
https://github.com/libgit2/libgit2/pull/6191
If everyone started using sha256 then all these problems would be addressed practically overnight.
- Change the binary file format in repos to support arbitrary hash algorithms, in a way which unambigously makes old software fail.
- Increment the Git major version number to 3.0
- Make the new version support both the old version repos and the new ones. Make it a per-repo config item that allows/disallows old/new hash formats. In theory, there's nothing wrong with having objects hashed with mixed algorithms as long as the software knows how to deal with that.
- The old format will probably have to be supported forever because of Linux.
Most user-facing utilities don't care what the hash algo actually is, they just use the hash as an opaque string.
And why is this an issue? Release the new version that can read new repo formats, but doesn't write them yet. Wait a year. Release new version that can write new repo formats and encourage users to upgrade.
Anyone who hasn't upgraded in the past year probably doesn't care about security and should be left behind. Besides, once they google the error message they'll figure it out soon enough. It's not like git is known for its great UX anyway.
That's an interesting idea, actually. I'm not sure they plan to support that, though? That would make things a lot easier on existing repositories; without support for mixed hashes, repos would have to have their history entirely rewritten, which would invalidate things like signed commits/tags.
there is one hash version, plus a translation table for the other format. no history rewrite.
new repos will use the new hash. old repos will eventually fully convert to the new hash, then all old hash links after the transition period will become obsolete.
Here’s the paper: https://marc-stevens.nl/research/papers/C13-S.pdf
(I have seen proofs of concept [1], but never actually heard of an exploit in the wild using it; for example, on: digital certificate signatures, email PGP/GPG signatures, software vendor signatures, software updates, ISO checksums, backup systems, deduplication systems, Git, etc.)
[1] https://shattered.io/
https://twitter.com/rauchg/status/834770508633694208 > a SHA-1 "Pinata" [...] claimed
https://news.ycombinator.com/item?id=13723892 > Make your own colliding PDFs
https://news.ycombinator.com/item?id=13917990 > Collision Detection
The most in the wild one i have ever heard of was when webkit accidentally broke their svn repo by checking in a collision.
However you can look at the history of md5 which had a similar flaw which was exploited by the flame malware.
https://en.m.wikipedia.org/wiki/Flame_(malware)
Either way, anything relying on hashes for data integrity should at least be flexible to the option of multiple hash algos. But with git, it's going to be hard enough as is to change to SHA-256, and I don't know how parametric it'll be.
Also, BLAKE3 is so significantly faster than it's probably worth waiting for (waiting for it to get vetted cryptanalysis-wise).
A new hash algorithm for Git https://lwn.net/Articles/811068/
Updating the Git protocol for SHA-256 https://lwn.net/Articles/823352/
I give -3 flying ducks about this, and don't want the Git storage format to be diddled with in any way. Git in 2122 should read and write a git repo made in 2010.
Git is not a public crypto system.
If you think a commit is important and needs to be signed, you need to sign the files and add the signature to the commit.
"Fossil started out using 160-bit SHA-1 hashes to identify check-ins, just as in Git. That changed in early 2017 when news of the SHAttered attack broke, demonstrating that SHA-1 collisions were now practical to create. Two weeks later, the creator of Fossil delivered a new release allowing a clean migration to 256-bit SHA-3 with full backwards compatibility to old SHA-1 based repositories. [...] Meanwhile, the Git community took until August 2018 to publish their first plan for solving the same problem by moving to SHA-256, a variant of the older SHA-2 algorithm. As of this writing in February 2020, that plan hasn't been implemented, as far as this author is aware, but there is now a competing SHA-256 based plan which requires complete repository conversion from SHA-1 to SHA-256, breaking all public hashes in the repo."
[0]: https://fossil-scm.org/home/doc/trunk/www/fossil-v-git.wiki#...
Joking aside, expected from a developer whose work is the recommended storage format for Library of Congress.
I’ve added to my todo list a reminder to raise this issue with mine. In fact, I’m going to give them a deadline for when we will start evaluating competitors that do support SHA256.
I suspect that most people on HN do not interact with their MS account team. That relationship is probably managed by your CIO or IT department. They probably have monthly or quarterly “business review” meetings. You should get this issue on the agenda of that meeting.
The article says "none of the Git hosting providers appear to be supporting SHA-256", and while GH is not mentioned by name (and I applaud them for indeed not strengthening this "git == github-the-brand" trap), I can't imagine GH was left out of scope when checking the major hosting providers.
Gitlab also appears to be lacking support [0], and the same with Gitea [1].
so it's a grey area where Git itself supports SHA-256-based repos, but without the major Git hosting services also supporting them, the support in core Git is somewhat useless.
0: https://gitlab.com/groups/gitlab-org/-/epics/794
1: https://github.com/go-gitea/gitea/issues/13794
Git basically supports it already. GitHub et al do not, and that is what is holding it back.
Luckily I've versioned the internal hash so the upgrade path back to sha256 should be as smooth as the downgrade was. I'm still bitter about it though.
Sounds like there's money in this.