Yes every repo is either one or the other but you fix that by rehashing the entire repo. Everyone can do this independently. It's entirely possible to maintain to identical repos in SHA1 and SHA256 mode but for the most part I suspect once updated people will simply pull down the new repo and use git 3.0 as a required version.
As migrations go, it's reading as simple to me. You'll just have to backpoint the commit signatures. I must assume there's a backwards compatible reference for them in git 3, right?
Or drop them and reference the old structure in a dire pinch.
Re-hash the entire repo as in rewriting all history? Hooo boy will that be a mess, I deal with things which reverence commits by hash in repos all the damn time. There are thousands of them in every Yocto project!
"The migration to the new format is simple; Just re-write everything in the new format, but also keep the old format around forever too since data is lost in the new format!"
Do you have any external references to any commits that matter, for example in your communication platforms (emails, Slack) or your bug tracker? Or, worse yet, in places where they aren't just text format references, but used for things like CI/CD caching decisions or security scans?
Once you rehash the entire repo, every single one of those external references will be broken. Because no, there's no support for looking up old hash -> new hash or the reverse.
I think I remember something for this for mercurial to git migrations, I hope when its git 2 to 3 something similar is made (or it will take some time to adapt like when python did its 2 to 3 migration).
There's a massive push right now from top down to have secure software supply chains. Google SBOM and SigStore. It's not an organic need but if you have government customers you don't have many options.
The difference with IPv6 adoption is that the internet relies heavily on network effects: so long as some hosts only have an IPv4 address, you need an IPv4 address for full connectivity, but then if everyone has an IPv4 address anyway, there is no immediate need to migrate to IPv6.
(Yes us Hacker News users have plenty of use cases for IPv6, like self-hosting and peer-to-peer networking and so on; we are not the average user.)
This effect doesn't exist for the Git migration. Each repo can be updated independently; it doesn't affect users of other repositories, and most likely, the majority of devs will work on some SHA-1 repos and some SHA-256 repos with no issue.
If anything, I would compare it with the Python 2 to Python 3 migration, which was also painful, but succeeded eventually (despite being much less necessary in the first place).
Which is considered a Really Bad Idea because tags aren't immutable, so there's absolutely zero guarantee that it'll point to the same commit a few months from now.
The GitHub Actions ecosystem found out the hard way, through some rather high-profile compromises. They hotfixed it by adding "immutable tags" to their platform, and are now working on adding a lockfile to... easily reference a commit hash.
I just started learning IPv6 with AWS since they charge $0.005/hr per IPv4. Maybe it will be more expensive in the future and eventually it will be the new default.
FWIW there's nothing about the addresses themselves that stops you subdividing a /64, but it depends on your router.
You could put your server at x::1 and static-route that address as a /128 on your router, if it supports it. The reverse route might be a bit tricky but putting ::0 on the router and telling the server it's a /127 should work. Anything outside of the /127 (so, all the randomly generated addresses on your home network) would go back through the router.
Now if that /64 is also changing every day, then it's a problem and idk what you'd do.
Github is THE main platform for git.
If github doesn't upgrade (and their code has been shit and hard to fix/update before) then the shift will not happen. Because yes, you can upgrade your repo independently, but if there is nowhere to push, no one will do it.
IPv6 is (also because of github) a great example for this. You can easily have an IPv6 address next to your IPv4 address, but a lot of websites (e.g. github) don't have that. Why would a normal company use IPv6 if even the bastion of nerds doesn't use it?
And to Python 2's "eventual migration" I can unhappily tell you, that my company (recently) bought an actively developed tool, that still uses Python 2.
I don't think GitHub is quite as important as that, outside of some specific projects that made bad/lazy decisions (thinking Golang here). If they didn't support 256 I think a lot of enterprises and open source stuff would simply jump ship to GitLab or elsewhere. In the enterprise world it only takes one person to write a security document banning sha1 for this to happen. For that reason, GitHub will support sha256.
I think you would be very unpleasantly surprised to see how many organizations deeply rely on GitHub. Think about how many open-source projects you've seen that have willingly tied themselves to GH-specific features, and then extrapolate to thousands of companies with masses of spaghetti code that are hopelessly reliant on their internal GH Enterprise features.
It could also be another Python 3 situation: backwards incompatible, unclear benefits with many downsides (3.0 and 3.1 being very slow), large number of libraries that need to catch up.
Does anybody know why it wasn't implemented in a backwards compatible manner?
One could wrap a whole merkle-tree with an additional extension tree, that just adds the new hashes. That way both kinds hashes could be used to traverse all data. The new hashes could be used to check the consistency, the old hashes would still be there to use in UIs or old release documentation. The downside being that you introduce more nodes in the overall data-structure which will have to be supported basically forever. And if SHA-256 is to week a third layer would need to be introduced. But the point is, it could be done. Albeit it would loose some of the elegance of the data structures involved.
Allowing both hash algorithms is, from a security standpoint, equivalent to just using the less-secure hash algorithm.
All repos need to end up using SHA-2 exclusively by the end. All tools that speak only SHA-1 need to be made incompatible intentionally. If the SHA-1/SHA-2 hybrid approach could allow that to happen, then it would be useful. If not, then it would just be a waste of time.
The way you would do that generally is to make it backwards compatible, then adding warning to legacy tools, then turning SHA-1 off by default, then removing it entirely.
Doing it in a backwards incompatible way creates a chicken and egg problem, can't convert repo to sha-256 because some tool doesn't support it, tools don't have an incentive to be updated because no repositories.
The point I'm making is that "backwards compatibility" cannot rely on SHA-1 support being around forever. At some point, all uses of SHA-1 must go. This includes not only the interfaces between Git and external tools, but also Git's internal data structures. Past a certain point in the near future, there cannot be a two-tier Merkle tree system anymore; the SHA-1 tier has to be removed before long.
2. If you can't use sha256, set the config to put things back to sha1. Wherever you need to do this you probably already set dozens of ENV vars or settings, just add a new one.
Or write a 15 page analysis about how the above is so hard people will probably just find it catastrophic to even think about.
It's easy for _one person_ to fix. It's not easy for the entire git ecosystem as a whole. GitHub, large internal corporate git repos, CI/CD systems, projects with submodules, etc. The second half the article explains all of this.
For the record, Y2K was not fud. It was very real, in a long list of datetime problems that are to come. Further datetime problems are coming at scheduled dates.
It was definitely FUD. There was a real problem (date counters would roll over), but the impacts of it were so ridiculously overstated that it eclipsed any sane discussion of the issue. We had people at the time predicting that planes would literally fall out of the sky when Y2k hit, which was never a realistic possibility.
That's the problem with people like you. When others take action to prevent tragedy, like with Y2K, you say it was FUD. When people don't take action, like with climate change, you say it wasn't important anyway considering it's not you who's dead. In summary, people like you are incapable of running a car, let alone civilization, considering you would even never the replace the necessary parts at the right times in your car.
Considering one of my team literally was carrying a Motorola satellite phone on the NYE/D of 2000-01-01 in case our mobile or land comms went down because the public transit was running all night, we took it very bloody seriously.
If there was an interruption to services on that night in particular, that's a serious public safety issue.
We had tested our system, and integration tested with all the other systems we connected to directly, but any FMECA analysis would show you that there were failure modes that we couldn't mitigate.
So people on planes falling out of the sky? Probably no.
People being crushed in a railway station on NYE? Possible yes.
> sha1 is broken in 10 years. Suddenly everyone has to switch all at once on the same day because it is a critical security issue
If you read the OP article, the entire point he's making is that this would never happen, because a hash algorithm being "broken" doesn't matter in practice, because true supply chain security has nothing to do with file hashes.
I don't understand why Git is not making the SHA-1 and SHA-256 modes far more compatible with each other.
SHA1-hashed objects should be able to refer to SHA-256-hashed objects, although this seems somewhat pointless.
But SHA-256-hashed objects should also be able to refer to SHA1-hashed objects, with a major caveat: if those objects themselves are part of a collision pair, then there is a genuine problem. But this is avoidable! Suppose that Linux decided to migrate to SHA-256. The upstream project could choose a pair of dates, say January 1 2027 and March 1 2027. Up to the first date, maintainers would be welcome to submit hashes of objects that are not yet in the repo but that they think they might submit later on, and, on that date, the upstream tree would finalize the list of these objects and reference it in the repo (with a new mechanism for this purpose). Effective the second date, the repo would start publishing SHA-256 commits and would never again accept a SHA1-hashed object that was not in the repo at the cutoff date or referenced as part of the Jan 1 block.
And now it would be impossible to get a new SHA1 collision in to the repo.
The only new git features needed would be:
a) actual compatibility so that a SHA-256-hashed object could reference a SHA1-hashed object
b) a new object type that's a list of allowed SHA1 hashes (or probably a tree of them) that is itself hashed with SHA-256 and a mechanism to link to one of these from a commit
c) a policy mechanism to set a repo to only allow SHA1-hashed-objects that a reachable from a preconfigured SHA-256-hashed commit
I skimmed the video, and I didn't quite catch that. Near the end of the video, however, she did mention that interop is in the works[0].
In any case, even if Git 3.0 were completely incompatible, it would suck, but it's not the end of the world. You just treat it as if you were migrating from one SCM system to another. CVS -> SVN -> Perforce -> Git -> Git 3.0 -> [...] been-there-done-that. This is something that both open-source and commercial projects have had to deal with over the years.
Or maybe it would be a repeat of Python 2.x -> 3.x. ¯\_(ツ)_/¯ With AI assistance, hopefully porting the tooling over may go a lot quicker and smoother.
My point is not that it's the end of the world (or the end of Git), but that it will be painful and unclear and confusing to lots of people. That would be fine if it made a huge difference in trust or protection, but it's the wrong way to do that.
They begin by saying mixing can never work but don't address the option of dual hashes... Then the interop plan is basically a secret git format that maintains both sha1 and sha256 hashes FOREVER.
I don't see why all users wouldn't want to keep both hashes.
Hm but the date is stored inside of the commit. The only way we can know that a commit's date is authentic... is through its hash. If I can forge commits with any SHA1 hash at will, I can make a repository whose head commit has the same SHA1 as the one in torvalds:
/linux but where any commit was replaced by a malicious commit with the same SHA1 and a fake date. You have no way to detect that my repo is inauthentic other than through a deep history comparison. The whole idea behind a merkle tree is that just checking the hash of the top is sufficient to know the identity of the whole tree.
I don't know what the solution is, but I'm inclined to believe that any repo with a single SHA1 commit is as weak as a repo with all SHA1 commits.
The date that a repo receives a commit is known to that repo. And a repo can stop accepting new SHA1 objects. And a SHA256 object could have a flag that says that no SHA1 objects may ever reference it.
The design of Git, as a Merkle tree, is meant to allow for use cases like this:
* I host a mirror of the Linux git repo.
* You download Linux from my mirror.
* You check out a commit, say fd179f8a05be3ccae366b9b96e176b51fbe54aab, which you know is a genuine commit through some out-of-band mechanism (mailing list, GitHub web interface, a line in a Nix file, whatever).
* You check whether the repository I gave you is legitimate or not by re-computing the hash of the commit which I claimed was fd179f8a05be3ccae366b9b96e176b51fbe54aab. If it comes out to be fd179f8a05be3ccae366b9b96e176b51fbe54aab, you know it's legitimate. If it doesn't, you know it's fake.
This is a completely normal use of Git. People download from mirrors all the time. People rely on commit hashes to identify a specific source tree. People trust that if whatever the mirror gave them hashes to the right value, it's genuine. That way, you don't have to trust the mirror.
If I can forge my own commits to have any hash I want, this whole model breaks down. I can replace some old commit in the repo with my own forged commit with the same hash, and when you download a copy of the Linux repo from my mirror, you'll receive a repo with malicious content, but it'll hash to the same fd179f8a05be3ccae366b9b96e176b51fbe54aab hash as a genuine repo would. This breaks the security model of Git.
> You check out a commit, say fd179f8a05be3ccae366b9b96e176b51fbe54aab, which you know is a genuine commit through some out-of-band mechanism
That's a 160 bit hash, which is SHA-1, which has the security properties of SHA-1.
Suppose you check out a commit with a given SHA-256 hash. That commit object represent the root of a tree where all the edges are hashes (and types, etc). I'm suggesting one of two designs:
a) (Simpler but weaker) If Linus has published that commit, then he is confident that he hasn't pulled in any too-new SHA-1 hashes and that there are no collisions present in what he thinks the tree is. So, by induction on the traversal depth, there is only one actual object identified by each edge, and those objects contain the hashes of their child edges, so those hashes are all correct.
This breaks if there is a malicious collision already in the tree.
b) (Stronger but higher overhead and more complex) There would be an object or objects, discoverable from the root by following only SHA-256 edges, that encode a duplicate-free mapping from SHA-1 hash to SHA-256 hash. The client finds and parses that and then, as it traverses the tree, each time it reads a SHA-1 hash, it computes the SHA-1 and SHA-256 hash of the referenced object, verifies that the pair is in the mapping and also verifies that the SHA-1 hash matches what the edge requires.
I think that (b) is genuinely cryptographically secure in the sense that, if you can construct a commit that has the same SHA-256 hash as an official upstream commit but different contents, then there is necessarily a SHA-256 collision.
For A), I don't understand what the point is? I never mentioned what Linus is confident about, I talked about what you can verify when you pull from my mirror. I could replace a commit from 2010 with a malicious one
For B), I would think this could work, but it's a completely different solution from what you proposed and what I responded to.
I think I stand by my second proposal. I also think it's absurd that, after all these years, upstream git still can't figure out a credible migration plan.
Maybe from your mirror. The collision attack that's been discovered is not a preimage attack though. It generates two colliding objects with the same uncontrollable hash, so you can't replace an existing hashed object, you have to sneak in one half of the pair.
More importantly we just shouldn't use your mirror if we don't trust you. If you're evil you're probably lying about all the tags and branch tips anyway.
If there is one way enforcement (i.e. there is one point where the last SHA1 commit was signed by first SHA256 commit), I think it should be safe ?
The "commit before" might be compromised, but the git commits refer a snapshot of a tree + a list of previous commit IDs, so the "new" SHA256 commit will not have any files altered
My reading of the docs is that when fetching/pushing from a SHA-256 repo, all objects are referred to (in the packfile fetched) by their SHA-256 names — then, when fetching, you locally compute the SHA-1 of each object for the translation table.
Presuming there’s some validation that SHA-1 names are unique, then that should be safe — the only way I can see one could do a pre-image attack is either fetching from a SHA-1 server (because then you don’t get the SHA-256 object name), which requires a second pre-image attack on SHA-1 (known to be feasible); or by having a second pre-image attack against both SHA-1 and SHA-256 simultaneously (and SHA-256 is still believed to be secure).
A lot of replies here seem to be asserting that this "isn't that hard" without addressing the thing that makes it most hard: submodule compatibility and the breadth of tooling
As linked by another commenter in this thread, Linus worked out years ago that even if someone inserted a malicious object into the kernel repo, it would at best be a nuisance and not a major concern.
Tools like git-filter-repo[1] support rewriting commit hashes in commit messages. git-filter-repo actually does it by default; see `--preserve-commit-hashes` in the manual[2].
... do not migrate old repos? I'm not sure why people would do that. Or, if they do, why would they replace the current repo name instead of creating a different one and keeping the old one closed to make the references work.
I don't think this is going to be a problem at all.
Unless the attacker was the one that made that original commit and message. The fact that SHA-1 is broken in specific instances means the algo is untrustworthy.
Once this starts being actual pain, we will each vibe the replacement index creator (git already supports replacement objects), for back-forth conversion, populated on pack and object indexing.
For massive perf and mem use damage. But oh well. And then we will wait for official version
There are plans to keep sha1s around in a database, but as far as I know, no way to transmit those, so they seem specific to individual forges. They can be recomputed, sure, but again, any signatures break and it's possible that in the case of an actual replacement, the recomputation is now wrong and not easily comparable. So what is the point?
This is not about forges, it is about the repo I have in a folder on my computer. The sha1 hashes shouldn't go away. Yes the forges also need to support this.
Note that there exists multiple ways to continue to lookup SHA1s in a SHA256 repo.
One example is to maintain git-replace refs for the rewritten SHAs but there also exist config flags to enable object format compatibility extensions that help translate the SHAs back and forth.
So they’ve been talking about this for many years, planning, and finally announce when they’re going to switch the default.
So this is the right time to post that everything they’re doing is wrong? Did you engage in all the discussions about it and how best to handle it? Whether SHA-256 was the best solution?
I don’t see anywhere that it talks about alternate proposals or why they might have been better. Why the particular suggestions here were rejected.
This seems like a bunch of Monday morning quarterbacking.
Ugh, I didn't know that SHA-1 submodules wouldn't be supported in SHA-256 repos. That changes the transition from painless to a major dumpster fire. Having to maintain converted forks, and use different hashes from upstream is going to be a mess.
The only reason C++ projects depend upon submodules is because C++ is just as archaic as COBOL these days.
I used to be one of the more vocal defenders of C++ but honestly, if the reason you’re defending git submodules is because there’s no better option in C++; then you’ve basically already lost the argument.
Literally every other language has found a better alternative. Even Go has, and that’s largely mocked for its prehistoric approach to modern programming language design.
>it will be an incomprehensibly expensive and ultimately valueless and avoidable global nightmare.
thought "costly" in the title and "incomprehensibly expensive" in the subheader meant this piece would discuss how much less performant sha-256 is on modern machines, but didn't see anything. isn't there hardware acceleration? how much worse is it?
Actually, I think sha-256 is possibly faster than the sha1dc variant that Git currently uses.
I just sent a patch series to the list that enables sha1dc to be accelerated on modern CPU architectures to close to normal SHA1 speeds, but since it was ported from a Rust project by an agent, it will never be applied.
How could SHA-512 be faster? It does more rounds of the exact same operations as SHA-256 with a bigger state. Although if it really were faster, SHA-512/256 gives you a truncated version.
It hashes a double amount of data per cycle with less than a double amount of operations.
SHA-512 is always faster in software than SHA-256, when run on 64-bit CPUs, and it is also faster in the CPUs that support both SHA-256 and SHA-512 in hardware.
Arm-based CPUs have supported SHA-512 already for many years and the latest Intel CPUs also support it, i.e. Lunar Lake, Arrow Lake S (S is for desktops, Arrow Lake H for laptops does not support it), Panther Lake and Clearwater Forest.
I expect that AMD Zen 6 should also support it, because they are the last important vendor without SHA-512 support.
All modern CPUs support SHA-256 in hardware, so it is faster when SHA-512 is not supported in hardware, otherwise SHA-512/256 is preferable, by being both faster and more secure.
Thanks, that makes sense. I was vaguely aware that SHA-256 halved the internal state from SHA-512 but not that it halved the block size, I haven't looked much into it. Although it makes sense in retrospect that they would simply use 32-bit variables to keep the same structure.
This article is full of mistakes and misleading claims:
1) It's claiming SHA1 insecurity is theoretical, while SHAttered from 2017 was specifically a pratical proof of concept. The only reason Git wasn't affected, is because they didn't bother bruteforcing a git-blob prefix.
2) It's claiming collision attacks don't matter, only second-preimage attacks do. This is incorrect, collision attacks are enough for code-smuggling problems, when two repositories are on the same git commit (verified by the full commit hash), yet contain different code in their git checkout.
3) The Linus quote "The real security is in distribution" is arguing that "git's content-addressed system should not be used to address content". It's arguing that, in case of curl|sh, you shouldn't use a sha256sum-gate to pin the content to something you've reviewed, you should instead ensure curl is fetching from an https server.
1) I link to the SHAttered paper, as well as Shambles. Git projects were not affected because it is an inefficient attack vector. I say it's impractical to exploit, which I think everyone agrees with.
2) I specifically argue that even if both attacks were practical and cheap, it's still not the problem we should be focusing on.
3) Have you read this email (that I linked to)? It is almost the same general message (20 years ago) that this blog post is. It literally goes though a theoretical object replacement attack and how dumb this scenario is and so SHA-1 is fine.
> 1) I link to the SHAttered paper, as well as Shambles. Git projects were not affected because it is an inefficient attack vector. I say it's impractical to exploit, which I think everyone agrees with.
It seems unlikely it will stay that way forever. Typically attacks get more efficient over time as researchers find improvements, not to mention computers getting better.
In 2015 it was estimated to cost $100,000, now the estimate is down to $10,000. Where will it be in 2035?
The XZ incident would have been so much worse if it would have involved a collision, pushing one object variant to github.com, and one variant to git.tukaani.org.
Then you would have security researchers making conflicting claims depending on which repository they first pulled from, even though they are on the same git commit hash.
Wait maybe I’m missing something, but are you saying the temporary confusion while people realize they have different versions would be the primary downside? I feel like that’s an acceptable loss. It seems likely that the data in both commits, while one healthy and the other corrupt, will still have to be very different in order to produce the same hash. It’s not like you can reasonably find same-hash commits that just change line19 execute_hack from true to false.
The XZ incident was using binary files checked into git. When you successfully collide this file, one with the trigger, one without the trigger, they get the same git blob-object hash (assuming the chosen-prefix includes the git blob-object header). Because of this, any git tree-object referring to this file is valid for either variant (no further collision needed). This tree-object can itself be part of another tree-object, which can be referred to by a commit-object. Signing this commit with ed25519+sha256 doesn't fix the fact that the tree-object underneath is ambiguous.
People assume sha1 git is cryptographically sound, and a git commit is a secure identifier to reason about source code, whether you and me like it or not.
In GitHub, fork networks are represented as one repo on disk (this is the main optimization that makes community pull requests possible at all). So you could fork a repo and then push an object with a has collision to it to change a file in the original, in theory.
the attack wouldn't work. Joe Schmoe? It's worse than just "being compromised"
You have repo of dependency locally, let's assume you downloaded good copy, the commits get compromised, you're safe.... right ?
Nope, if there is build server along the way and ESPECIALLY if it practices building from clean state every time, the build might be infected while your local copy is clean, giving no chance to notice it, unless your entire chain including local builds are reproductible AND you actually check it
In agreement that this is good old fashioned cargo-cult security theatre, but tom7 also coined a more catchy phrase for this, he calls it "toxic max-security." http://tom7.org/httpv/httpv.pdf
Conveniently Tom didn’t mention anything about Edward Snowden and what he published. That was basically start of TLS everywhere.
Then he didn’t mention ISP idiots that were actually injecting ads to cute websites like Tom’s. I hope Tom likes when his website is used by ISP to make money on ads he doesn’t have any control over.
Then he goes on to criticize certificate transparency, but it works. Companies got kicked out from trusted root program because they were doing stupid stuff like making certs they shouldn’t.
Let’s not forget glorious state of Kazakhstan where without TLS they would just listen to all traffic - well with TLS they were trying to pull MITM but were uncovered and got their stuff removed by TLS ecosystem.
If I recall, tom7 was at odds with chrome throwing up a warning at users trying to visit his website because he didn't support TLS on it. He wasn't against TLS. Calling them a TLS antivaxxer is not accurate.
While he does indeed have extensive knowledge of TLS/SSL. He still completely side steps points I wrote about and exaggerated many minor inconveniences. While PDF seems quite up to date it also picks on stuff that is not there anymore like green padlocks.
I don't particularly like the terminology but, still, the attitude that TLS is just for "sensitive" websites, actions, or data is quite wrong. Even if you don't care about surveillance or ISP ad injection, you should care about infrastructure compromises. It sounds exotic and high effort but it's not. Unless you're personally auditing the coffee shop, airport, library, and/or hotel Wi-Fi hardware and network before you connect, it could affect you easily. It's not even a choice that reasonably belongs in the hands of website owners, because they don't control all the paths from users back to them. And residential and municipal ISP networks can get compromised, too.
Yeah I kind of forgot exploit injection in the spur of the moment as of course I rushed to correct "someone on the internet".
ISP injecting ads is like annoying but if someone knows as much as Tom about TLS and totally skips rouge "airport wi-fi" can use his website to own someones else device that is the argument I should use for calling him or anyone else names on the internet.
I guess Kazakhstan example kind of covers it, because I do believe they would definetly deliver malicious payloads to dissidents.
There's a lot of lingering "SSL is just for your bank" sentiment out there. There are other ways to achieve integrity protection than using TLS, and there are even ways to use TLS without WebPKI, but nobody is putting any effort into any of them, and so TLS+WebPKI is the solution, and no website is exempt, because it is fundamentally an infrastructure problem.
Did you opt-in to having your content shown in "previews" and Gemini "instant answers" along with Google ads?
I could just as easily argue that by paying my ISP and signing up to their T&Cs, I've opted in to seeing their ads, not the ads from some random website.
What ISPs were doing they were injecting ads and you had no way of knowing it happens unless some visitor was annoyed enough to make you aware by sending you a screenshot.
With Google you have to make an agreement and put piece of code in your website willingly and then you get minimal cut but still, that is totally your choice.
I say it's impractical to exploit, which I think everyone agrees with.
Impractical for an individual, definitely. For a large org, maybe, but if the payoff was big enough? For a nation state level actor intent on doing something, absolutely not.
The go-to example is Stuxnet. Some countries wanted to attack Iran's nuclear enrichment programme, so they spent 5 years developing a worm that used multiple zero day exploits to attack a specific controller in a specific model of gas centrifuge. Could Mythos write Stuxnet? Unlikely, but a knowledgable team with access to it could probably write it in a lot less than 5 years.
'impractical' has very different values for different groups.
Okay, fair enough.
But does that make sense as a default setting then?
I can see that some things might have a risk profile that might possibly make all this costs still worth it, but does it make sense to have these unicorn projects effectively blow up 20 years of ecosystem?
Shouldn't the extra cost of doing something out of the ordinary be carried by whoever does something out of the ordinary?
This feels like a bridge to be crossed when one gets there (if at all).
__
FWIW, we actually do have a choice here. No one is forcing the industry at large to adopt an unpatched git 3.0 binary built from a source that makes that a default.
This should be a trivial overlay to carry around with effectively no downsides. So convincing whoever is steering that ship doesn't necessarily matter, as long as enough sane pragmatics agree on how defaults should actually be.
We are migrating all of PKI to the more costly and less efficient postquantum cryptography, even though nobody will reasonably use a quantum computer to snoop on your home IoT daily reports. I mean, I assume that what you are doing on your free time is not worth governmental attention.
The rationale of mass migration is that if you don't impose it, nobody migrates. This has notably been the case with famously insecure SSL parameters (512 bits RSA keys, PKCSv1.5...). And many companies may believe they are not critical, which might be true until it is not.
Case in point: you manufacture walkie talkies and suddenly your products have bombs inside. Or you maintain a compression library for free and suddenly you are shipping a backdoor to all Linux products.
Sure, but here I guess the main idea is that everything is linked, especially how libraries package managers work nowadays.
In order to compromise the big player, you only have to compromise the weakest link in its supply chain. In effect that means that leaving the migration optional is as useless as doing nothing.
As for "nobody will reasonably use a quantum computer to snoop on your home IoT daily reports", they already have access to your data one way or another, so really no need for a quantum computer. :P
I don’t understand your comparison with walkie talkies. What are you trying to highlight here? In the case of the pager attack in Lebanon the devices were booby-trapped. Are you trying to say that the device designers should have somehow done something about it? I cannot really make sense of it
My point is that the business of pagers is rarely seen as something likely to be booby trapped by a nation state. Yet in the context of Lebanon it happened.
I don't remember where the compromise happened (factory, distribution, sell point), but very clearly at least one of them did not expect to be a critical asset in the Israel - Lebanon war.
Exactly. Reasonably speaking, nobody tries to piece encryption at all, ever. That doesn't mean we should stick with 3DES. Reasonably speaking, nobody will try to break in to your house at all, ever. That doesn't mean you shouldn't lock your door. Security is already about protecting against infrequent events.
> And many companies may believe they are not critical, which might be true until it is not.
I'm at a K-12 public school. That shouldn't be on the front lines of a war with Iran, but, in cybersecurity terms, we are. If you disrupt a school district, you disrupt one of the largest employers in the area. You also disrupt the largest childcare facility in the area. The amount of economic damage you could inflict on a community by disrupting the public school system is pretty extreme compared to the amount of funding provided to protect it.
Yes, and in the days of major supply chain attacks and state-sponsored near catastrophes like xz, "it's probably good enough" starts to look incredibly naive.
Linus's "what matters is distribution" comment also doesn't make sense when merge effectively is distribute. Which, again, is the reality of supply chain.
Stuxnet is essentially "boutique malware". You can buy it/have it built with enough money/resources.
Weaponizing a cryptographic algorithm with some theoretical vulnerabilities (but no by-design backdoor built-in) is a totally different game. And TFA is right that it's a dumb endeavour. You can probably "stuxnet" your way in for much cheaper.
> Impractical for an individual, definitely. For a large org, maybe, but if the payoff was big enough? For a nation state level actor intent on doing something, absolutely not.
In all the years since 2017, with all the orgs having huge GPU-filled data centers (and’s an interest in software security) has anyone demonstrated a real git collision?
No, it really is the correct word. They just pick a huge random number. There's no value in "serializing" or "fingerprinting" such a number. It has two purposes, we need it to be unique, and we need it to be random and because it's a huge number being random means it is unique so we are done.
yeah i don't know as esoteric as it is it could be a referential index into a bitmask/seive for your key.
there's a lot you can do behind the scenes with even a couple bits of... free space?
the key should be enough of a unique id to not have to require this? see (hash). a separate linked 'randomized' unique identifier might be crazy man territory but i'm not lying... your 'weakened' key doesn't _need_ to be all zeros, just predictable eg. within a certain time frame, anything that can reduce the search space is dangerous.
edit: it could contain an identifier for which hrng was used to produce it.
This feels like you either don't know why we require this or, perhaps even more weirdly, you do understand why we require this but you've concocted some elaborate conspiracy to justify that rather than accepting the obvious explanation.
At some point "The world is actually ball shaped" just makes a lot more sense than the thousands of years of vast elaborate conspiracies to keep you from realising that there are Mole People whose underground civilisation is accessible from Antarctica. So in the hopes that it's the former (you just didn't understand), I shall endeavour to explain.
The MD-series and SHA-1 and SHA-2 series of cryptographic checksums use what is called Merkle–Damgård construction which operates on fixed sized blocks of data. In this design if we can find a collision before a certain block, everything after that point will keep colliding. The converse doesn't work, there aren't suffix collisions which would work for any prefix, only prefix collisions which work for any suffix. In SHA-3 and newer hashes a "Sponge" construction is used, we just pour stuff into the sponge and only squeeze out a fixed-size hash at the end, so these "prefix" attacks would need to change the entire state of that sponge.
An X.509 certificate's serial number is very, very early, before any of the information which can be chosen by the recipient. So by ensuring this number is entirely random we're making it impossible to construct information which results in a collision, the prefix for their input will be random so it's now impossible.
This is a "defence in depth" strategy. The SHA-256 hashes used are believed to be fine, for the immediate future, but even if they were vulnerable to a prefix attack as we know SHA-1 is, the choice to have random serial numbers defends us anyway, the attack wouldn't work on certificates.
Impractical for everyone. Because if you want to do this, there are better attack vectors, even if second preimage was easy, even though it is impossible.
And it’s also the question of different use cases.
Asking to trust in an authority (while the main authority Microsoft/GitHub has is essentially figuring out enterprise sales well enough to be acquired by a company desperately needing developers after fumbling badly in the 2000s) is exactly the opposite of my stance - it is a large corporation, with heavy employee rotation, with substantial exposure to various forms of regulatory pressure and to various forms of corruption.
Which is why the cryptography exists to prove the developer-to-consumer trust without trusting the intermediaries. Yes, I do check GPG signatures. Yes, I do include a git commit hash in the binaries I build. And surely I want to make sure that this doesn’t mutate because some unknown engineer at GitHub had a bad case of gambling debt.
When I check out code from a git repository in a pipeline using a git hash, I expect the code to be exactly what has been reviewed by me under that hash.
Everything else would just be a crazy invitation to make supply chain attacks uncircumventable.
Dealing with potentially-hostile hosts is quite common, actually. See for example how most Linux mirrors work, or Subresource Integrity with HTML.
Turns out securing a service to transfer a single hash is a lot easier than securing a service to transfer gigabytes of data.
Even if I don't fully trust Github, it is still incredibly convenient to be able to upload my code there and then send someone an email telling them to fetch commit `123abc` from some repo link. As long as my email isn't compromised, that should be secure.
Similarly, if you push to github, then trigger CI through something other than github actions, then (other than the SHA-1 vector), you have reasonable assurances that someone who has compromised GH cannot compromise your CI host.
Code signing also generally relies on hashes. You don’t encrypt the code to make a signature, you encrypt a fingerprint of the code, which is usually a hash from the SHA family.
I pushed aviation toward starting with SHA-256 instead of SHA-1 for code signing more than fifteen years ago, just after NIST first started discouraging the use of SHA-1 in new code.
Rumor has it that GitHub has a flat namespace for commits. They don't store "user1/repo1/abcd1234" in one file and "user2/repo2/abcd1234" in another. Both references point to the same commit in a global shared space. If the hashes are truly unique, then that never matters, because the odds are approximately 0.000000000...000 of you and I accidentally generating the same commit. However, if I see that you pushed commit abcd134, and then I can build and push a colliding commit, and the backend doesn't check uniqueness before writes because the odds are infinitesimal that it'd ever matter, than voila, I've updated your repo by writing to my own.
Or if first writer wins, and I know that you have a popular non-GitHub repo that you're about to migrate into it, then I could pre-poison the namespace by writing my own version of a commit that I see you already have in Codeberg or Savannah or wherever.
I don't swear that this is how GitHub actually works, but I've had knowledgeable friends swear up and down that it is. And honestly, it'd make sense. They could shard storage by the first 4 digits of the hash or something, and that'd be vastly more efficient if all commits were writing to the same space.
That's per-repo. The claim is that user/repo/commit is flattened to just the commit. Your counterpoint (which I agree with and is well-known) is that it's flattened to repo/commit (dropping the user).
Ah right, though I think it is rather unlikely Github flattens across multiple repos given that Github has already blogged about how their repo storage backend (Spokes) works, and it seems to operate entirely on whole repositories.
So, I haven't worked at GitHub in some time, but we never had a flat namespace for objects. There are a lot of SHAs in the DB but objects are always namespaced by repository. Forks shared an object database for efficiency, and have technically added reachable objects to a shared database via fork, but it's never been a real problem afaik.
But the very wrong assumption here is: "if I see that you pushed commit abcd134, and then I can build and push a colliding commit, and the backend doesn't check uniqueness before writes"
All parts of this are incorrect.
You can push a colliding commit to a fork, but Git will see that it's already there and ignore it - first write does win. Also, "backend doesn't check uniqueness" is also wrong. The server will check for collisions and if this particular case happens, the server will see this and warn you _AND_ not write the object.
> I don't swear that this is how GitHub actually works, but I've had knowledgeable friends swear up and down that it is.
All the details on how GitHub's infrastructure has evolved over the years are very publicly detailed in the GitHub engineering blog and in technical talks. There are no "secrets" or "rumors" here, all the information is one google search away. Perhaps you need to re-evaluate your priors on how knowledgeable your friends are.
> And so if you don't trust the server that is hosted on or the security of the transport mechanism like TLS/SSL, such that the content may be manipulated by adversaries, you think that git hashes are good enough?
Yes, they ought to be good enough. That has always been git's security model.
Note that the commit doesn't have to be communicated over the same channel as the git data.
> Well, what about someone who is fetching the commit from that server for the first time and has nothing to compare the hash against?
Then they're vulnerable. But what about somebody learning about the trusted hash in another way, e.g. a build server getting an internal call authenticated by an authorized developer?
Just because you can think of an insecure way to use git hashes doesn't mean there aren't any other, secure ones.
You are not saying anything other than that hashes are useful for signing. When you communicate to someone that you trust such and such a hash, and they should too, you are informally certifying the hash, the cryptographically formal way to do which is to sign it.
> You are not saying anything other than that hashes are useful for signing.
Yes, but you seem to be saying they are not, or maybe just that git hashes are somehow inherently not capable of taking that role, unless I'm misunderstanding.
> And so if you don't trust the server that is hosted on or the security of the transport mechanism like TLS/SSL, such that the content may be manipulated by adversaries
Wait, why is anyone expecting that to be a good idea at all?
> And so if you don't trust the server that is hosted on or the security of the transport mechanism like TLS/SSL, such that the content may be manipulated by adversaries, you think that git hashes are good enough?
How does this matter? When I have a machine that I trust and a git hash that I trust, I don't need to rely on the transport mechanism. As long as the content cannot be forged to match the hash, the transport is completely irrelevant.
Sorry, no. For trusting code there is code signing. A sha-1 hash is cryptographically the wrong approach for it or we wouldn't have RSA and ECDSA algos.
The reason why not is very simple: the SHA-1 algorithm was chosen because it made it practically impossible for two commits to have the same hash, not on the assumption that it would remain forever immune to collision attacks.
>I expect the code to be exactly what has been reviewed by me under that hash
Sorry, no.
Again, hashes are by definition, insecure. I.e. you don't have a guarantee that the hash references the same commit, just a (very, very strong) probability that it does.
> Everything else would just be a crazy invitation to make supply chain attacks uncircumventable.
What? Uncircumventable? Logically equivalent, I read your statement as "So if all cars are not blue then they must be red"? This does not follow...
I like the last part of the article, which proposes a (very reasonable sounding) extension for people who care (more) about their code-sec. You should be able to swap out your hash algo without having to rebuild your content addressing system.
> Again, hashes are by definition, insecure. I.e. you don't have a guarantee that the hash references the same commit, just a (very, very strong) probability that it does.
This doesn't seem to be a useful definition. Would you classify every computable algorithm as insecure, because by generating a random bitstring, there is a (very, very low) probability of guessing the hash/secret key/solution/signature?
> Again, hashes are by definition, insecure. I.e. you don't have a guarantee that the hash references the same commit, just a (very, very strong) probability that it does.
Cool, you've just defined the foundation of signatures and web encryption as insecure. What next?
You could pull a malicious colliding PR and reject it. Then you pull the other half of the collision without realising it is, and it's something good and you merge it. But your CI server already saw the malicious one and thinks it's the same, so builds the malicious code
That's what a lot of the responses are missing, which is why the response to them in turn is both no and yes. There's no published threat model that I know of for the properties that the hashes are supposed to be providing for git, which means anyone can make up any required property they like and then confidently state that SHA-1 provides or does not provide it, see this discussion for examples. The result is, to use another Linus quote as the OP has already quoted him in the writeup, "people wanking around with their opinions".
We use SHA-1 in our storage mechanism, which predates git. There is a (quite long) written threat model. Someone being able to generate collisions with an enormous amount of effort under just the right conditions is not a threat under that model.
No, that argument misses the point. Cryptographic hashes enable you to trust, that I only have to review the changes since the last trusted commit. That is more important for the developers and maintainers of the software and less for the users.
In order to work for the users, a thing first has to work for its makers.
There are a lot of tools where the lines blur between “for the users” and “for the development team” because the users benefit from some things that make the developers’ lives easier.
> Git hashes are not supposed to be a security mechanism.
They absolutely are, both when you're using signed commits (i.e. GPG, SSH, and S/MIME) or just identifying a given commit/repository state by hash via a secure channel and then serving object data over untrusted transports.
It's entirely possible that you don't use either, but that's certainly not true for everybody.
Calling anyone using either to be "doing it wrong" is borderline gaslighting: Git used to have these security guarantees, and just because they're now broken doesn't mean they were never there in the first place, or that it was stupid to rely on them.
Exactly! The recommended way to use GitHub actions is to use their deploy hash as the version to prevent hijackings. It was never the intention (nor would this have been envisioned when git was created), but security needs to be applied to the way tech is used.
Package managers even use it. E.g. you can have a cargo dependency pointing to GitHub at a specific commit. It's definitely intended to provide end to end security without depending on GitHub being secure.
The important point is that the switch is breaking backwards compatibility. The proposed solution, a new independent hash just for verification makes sense.
This is the top comment and yet says absolutely nothing to refute anything in the article, merely declaring that it's "full of mistakes". I feel like people haven't read the article, or this comment, and are essentially religiously predisposed to so-called progress, no matter the cost.
You are basically resolving a non-existent security problem by generating a far bigger security problem, because I'm 100% sure that a ton of software just assumes that a git commit hash fits in a `char[40]` and thus will buffer overflow like hell if they try to operate on new repositories.
And we are talking about who knows how many tools that work with git built in the years, and this is also made it worse from the fact that most tools just invoke the git binary and capture its output instead of passing from a library.
I like more the solution proposed at the end of the article, do not change sha-1 but instead, if you are relying on git commit for security purposes (that was never the intended use) add another header to the git object with a sha-256, so that with the small expense of computing the hash twice you don't break 20 years of existing tools that make the assumption of the git commit being 40 character long.
Persistent problems with increased hash lengths seem unlikely: if bad git clients have erroneous truncations or buffer overflows, they can either fix them instantly, problem solved (they have had many years to implement and test SHA-256), or fail to fix them and be written off as obsolete and incompatible: problem equally solved.
I would have liked sha-256 truncated to 40 characters (by analogy to sha-512/256 that'd be sha-256/160). Cryptographically that's probably fine. But 160 bits is not a lot. And probably fine doesn't tend to inspire confidence in the field of cryptography, it's not a very well studied scheme
An advantage of a hash with a different length is that a full length sha1 commit hash and a full-length sha256 commit hash can't be confused for each other
Your points are full of mistakes and misleading claims.
I don't simply mean to be disparaging - its important to security that the people making the decisions are a) competent b) can read, otherwise any "security" decisions they are making are at best probably insecure, and at worst, causing active harm and insecurity, DOS, etc...
Generally, I think you didn't read (or at least comprehend) the article:
1) Your assertion is factually incorrect. Practical proof of concepts are not "CVEs exploited in the wild". The article is not claiming that SHAttered is not correct, it in fact references it.
2) I don't think you read the article. The article claims the exact opposite, and in fact addresses the issue WRT to distribution.
3) Your sentence here is very confusing - I'm going to give you the benefit of the doubt and presume that you mean to say that the problem here is in the security of the naming. HTTPS itself has nothing to do with distribution security, that would be DNS/SecDNS (IFF you are using git+http protocol, then HTTPS is relevant, but not to git otherwise). But this is exactly what the article was talking about, the distribution is the security, not the hashing algorithm.
The first reason the author lists for why this will be bad is only an "issue" on Git hosts that don't allow repo creation on push (which is brain dead of GitHub). Any other host, you push your new repo, and it will see the hashing algorithm, and receive the contents accordingly.
Submodules is a legitimate argument against this, though I don't know how widely this feature is actually used, and similar to the arguments in favor of switching the default branch from master to main, this is simply a setting which can be changed.
I do like the idea of commits having both hashes, and am surprised that idea has not been explored further.
Generally though, I think the author's strongest argument is simply that the change isn't strictly "needed", and all the other issues presented aren't the strongest arguments against change.
It was theoretical - the point was that maybe some paper is published or some new tech or issue comes up. Now we have to do this again. If we separate the concerns, then we don't have to deal with both as though they're one problem. We can deal with one thing for content addressing and another for trust and security.
Heard. If we’re going through this pain of a migration- adopting something that is forward compatible at the ecosystem/community level like ipfs makes sense to me. You throw a byte at the front of the hash which indicates the format the hash is in.
We have a content addressable storage system internally and we use a self describing hash for it, so we can change the hashing scheme for different use cases.
Can someone more cyber-pilled than me explain what the actual risk with Git hashes being susceptible to collision attacks is? Obviously accidental collisions are problematic, but to my understanding the probability of that is still approximately zero.
Best I can tell, all a forced collision would do is let someone who already has control of a repo modify the history in a far from plausibly deniable way. Which in practical terms, they already could do simply by replacing the whole thing, because who's out here using git hashes as a security tool? Every pinning I've ever seen has been to tags (which can be modified at will), or hashes of the actual payload (which doesn't need to be the same as what git uses).
schacon: Really like the "Independent Tree Hash Headers" idea.
How difficult would this be to get this functionality into git?
Would it cause any breaking changes with older versions?
Have you discussed this with any git devs to see if they are open to adding it?
Actually, this entire blog post came out of a short chat at Git Merge a few weeks ago with Jeff King. I argued more or less this and he didn't _entirely_ disagree, though he has good counterarguments on the list over the last few years, so I don't really know how he thinks about it ultimately.
I would write this to the mailing list, but I thought a conversation that includes people outside that list is more interesting to me. Ultimately I'm not sure if I'm dumb about this or the whistle blower that's willing to actually say "maybe this isn't the right call"
I thought this would be a snark but it's an extremely well put together argument against the "Hashmageddon".
If you're replacing the weakness of SHA-1 just by going to another algorithm, you better be prepared to go to the next one when sha256 collisions happen, and it doesn't sound like git's design would be easy to modify for this type of crypto agility.
I do like their proposal for using signatures to establish trust and allow swapping sha256 for whatever comes next.
Technically, git's design (thanks to very smart people trying to solve this problem like brian and others) is _very_ easy to modify to different hashing algorithms now. A lot of amazing work has gone into this in recent years.
However, it's not a git problem. It's an ecosystem problem. It's that every git repo has to choose one and they're entirely incompatible with each other. That is the cost and the difficulty.
I think there is a question though when that will happen and if it will be in our lifetime. SHA-1 started showing weakness in 2005 (collusion in 2^69), the sane year git was invented. Nobody has found a similar weakness in SHA-256 as of yet.
It took 20 years to go from vulnerability in sha-1 to having to replace it out of caution. There is no such vuln in sha-256 yet. It could easily be 25 years before we find one, and another 25 years before we have to do something about it. Will git still be used 50 years from now?
> All it takes is just one collision to consider it broken right?
No, its considered broken before that stage. i.e. when someone discovers an attack that would allow someone to create a collision faster than they should while still being impractical.
> With the kind of compute power available nowadays and AI models I wouldn't be surprised we see it much sooner.
Computer power doesn't super matter, what matters is algorithmic breakthroughs. So far i dont think there are any examples of major breakthroughs of that type via AI, although perhaps i am just misinformed. Its still early in the AI revolution, it might still happen, but as it stands i don't think there is any reason to worry about that.
One of my favorite fun facts about Fossil SCM (another source control by the devs of sqlite) is that they patched their use of SHA1 6 days after the shattered attack was published:
"Both Fossil and Git started out using only SHA1 hashes. But when the SHAttered attack against SHA1 was published on 2017-02-23, the need to migrate to a stronger hash algorithm was recognized. Fossil added the ability to use SHA3-256 as an alternative on 2017-03-01 (six days after the SHAttered attack was first published). SHA3-256 is now the default for all new repositories and check-ins in Fossil, though older check-ins that occurred prior to SHAttered can still use their original SHA1 hash. Hence, no repositories had to be rebuilt and no hyperlinks were broken."
To me it's so interesting watching in realtime Git is still battling with this decision and for Fossil it was just another week of development.
That whole page is fun to read. Another fun fact somewhere else in the docs is that Fossil uses a grow-only set to store check-ins. They came up with this scheme some years before it was formalized by CRDTs!
I mean, there are two things here. One is how difficult it is to have a different hashing mechanism. Brian and other heroes in the Git core group have done amazing work to make this _technically_ possible on a repo level. To test some of my theories, I trivially implemented MD5 and an insanely dumb and easily breakable hash backend. It's not _hard_ to change the mechanism now. It's about the community.
Fossil isn't difficult to change not because it's technically harder for Git but because Git has a community and ecosystem that Fossil does not. The cost is not in the individual project for Git, the cost is because there is _so much_ in Git and this bifurcates everything.
As an expansion on that, as far as I know there's only one implementation of Fossil. There was at least a Java one for git, I believe, and I don't think upstream git is related to libgit2 either. And GitHub is doing… something not visible from the outside.
(… looking at the parent, though, I imagine there might be some information from the inside…)
> As an expansion on that, as far as I know there's only one implementation of Fossil.
That's is, since only recently, no longer strictly true: the age-old libfossil recently got client sync support, so is now (aside from _serving_ repos) essentially a standalone impl (its own developer still uses fossil(1) stash, patch, and diff -tk features, but otherwise uses libfossil's counterparts).
Also, Dan Mestas has <https://github.com/danmestas/go-libfossil>, a Go library for working and fossil, and he is also working on <https://zeitforge.app/>, a clean-room impl. of fossil (whereas libfossil is largely ported directly from fossil(1)) which even goes so far as to _not_ use an sqlite database for its file storage.
Dan Mestas and Dan Shearer are working on finalizing RFCs for fossil's sync protocol and artifact format, and zeitforge is created by carefully managing LLMs which are reading that draft (but not the source code of libfossil or fossil).
I personally have been using `go-libfossil` in a project! It's got some sharp edges but insanely useful in avoiding CGO and using fossil at the same time
It's easier to make world breaking changes when the world is really small. If git could magically just get everything and everyone to cut over and use git 3.0 in a magic instant, it wouldn't be having this problem.
That's not true because of extended support, but even if it was true, what's the goal here? What benefit does the entire ecosystem share from getting people standardized on 11? Botnets get 25% smaller? That's not some kind of important threshold. Or developers can count on access to a newer mutex API or something? Whereas getting everyone on the same git hash or python or IP protocol streamlines things enormously.
> Whereas getting everyone on the same git hash or python or IP protocol streamlines things enormously.
You don't think that if we had everyone on the same Windows version we could streamline things enormously?
For MS, for developers, for hardware manufacturers, and so on to be able to deploy, for example, ipv6 since everyone is on the same stack? Same stack, same bugs, same fixes.
Microsoft can already put out an update to both windows 10 and 11 at the same time. They occasionally do this for bugfixes. That hits basically all Windows users.
The difference in effort between updating one versus two very similar code bases is minor, especially when "Windows 10" and "Windows 11" each refer to multiple versions already.
For drivers you only need one version to support both.
IPv6 has been built into Windows for ages. If you think they can force toggle it or something, that won't work at all and deleting Windows 10 wouldn't make it easier.
I can't think of a situation where consolidating Windows versions makes a big difference. I can think of useful situations based on getting rid of all other OSes and/or Microsoft instantly forcing out updates to everyone, but those are different scenarios and they come with severe downsides.
So you're there's exactly 0 effort in managing things for multiple Windows versions, for anyone along the entirety of the ecosystem? From MS developer, third party developer, hardware, drivers, users, etc?
How about testing? Do you think that there's no testing put out when a fix has to go out for 2 OS, regardless of how similar they are?
> What problems are solved by getting all Windows users onto Windows 11 in particular?
I have given a few. If you don't want to see it, that's fine.
Isn't sha1-dc just checking for a small list of hard coded disturbance vectors? Meaning that script kiddies can't easily reuse exact values that researchers spent a bunch of time precomputing. That's far from being a good long term solution.
I'm not an expert on this but my impression is its a bit in the middle. It isn't a great long term solution, but finding new disturbance vectors is very hard. Its not like someone can just go find more on a whim.
I always liked the ipfs concept of a multihash where a hash algorithm id is stored with the hash, I don't know how well it worked in practice, but in theory all applications of the protocol are now forced into a world where there are multiple hash formats and it can and will change.
We don't need that. Introducing an unknown hash algorithm itself is a security issue.
This problem isn't hard or new. Just look at things like a TLS handshake. You need to separate the protocol from the storage implementation.
I personally believe the initial Git programmers were too in love with the efficiency of doing a bitwise 160 bit comparison on the stack and they sacrificed the known issue of changing the algorithm to do it. A decade earlier the same thing had happened with MD5.
The post's argument that hash collisions are irrelevant in practice is not convincing at all. Basically they amount to:
1. Collisions aren't as bad as preimage attacks
2. Even if you made a file-with-malicious-hash, how would you get people to pull it?
3. Other attacks are a bigger problem (social engineering)
(2) is laughable in a world with github. It's common for unknown people to submit pull requests to code bases, and for those changes to be reviewed and merged. For example, as part of reviewing pull requests, I have `git fetch`'d proposed changes to my local machine to check behavior on some additional test cases. "If you fetch it you're fucked" is unacceptable as a security boundary.
(1) and (3) are just tu-quoque arguments about other attacks being worse. The relevant question isn't how bad other attacks are, it's how bad this attack is.
The fundamental problem with collisions is that software often assumes they can't happen (or is not tested against them). Thus collisions can trigger bugs, or otherwise cause surprising behavior. For example, webkit figured the colliding PDFs demonstrating a sha1 collision would be excellent for unit tests, so they merged the PDFs into their SVN repo... which completely fucked it [1]. I don't know the exact internals of git so I can't comment on how you would get surprising things to happen, but "oops the file you merged was different than the file you reviewed" and "oops the repository got corrupted" seem entirely plausible.
It doesn't reject, but it will not replace. Same for a fetch/pull. That is another issue with this attack vector (that Linus also mentions) - it has to be the _first_ time that a node has seen this object. It makes the attack even more difficult than it already is (in like 4 different major ways)
(2 counter) is impractical because all nodes of git will not replace objects if it thinks it already has it. So any attack has to assume this is the first time the node fetched, which is difficult before trust is established, which is difficult. This is part of the argument Linus originally outlined for this vector, which is that it only works for _very recent_ objects.
(1/3 counter) is not what I argued. I argued from the worst-case position that collision and preimages were theoretically cheap and fast. Even in that case, I feel my arguments hold.
The main issue here is that you assume you can replace an existing object with a replaced one, which you cannot. Not only that, but in all known cases, the sha1dc variant of SHA1 that Git uses will even _tell_ you that someone tried to do this, which singles out the source quickly.
> but the point is the SHA-1, as far as Git is concerned, isn't even a security feature. It's purely a consistency check. The security parts are elsewhere, so a lot of people assume that since Git uses SHA-1 and SHA-1 is used for cryptographically secure stuff, they think that, Okay, it's a huge security feature. It has nothing at all to do with security, it's just the best hash you can get. ... [1]
Have you seen algorithm-agnostic protocols, like TLS v1.2 and IPsec? It always turns out to be a bad idea. Future upgradability is fine but you really don't want implementations of the protocol to be incompatible with each other and you don't want attackers to have any chance of tricking you to using insecure protocols.
Flexibility made sense in 1995 when nobody was sure which algorithms would stand the test of time. Even in 2005 it was unnecessary and in 2015 it was an outright liability.
Your argument is weird because it assumes we can know if a hash function will be secure forever. Assuming git would use SHA1 forever seems shortsighted.
It's of course possible that SHA-1 was not originally intended as a security feature, but according to Hyrum's law, every observable API behavior becomes something that somebody starts to depend on, so if you start very publicly shipping a cryptographically secure hash, you better keep it cryptographically secure.
At the very latest, this fact was cemented when first-party git commit signatures started depending on the security properties of SHA-1.
> if you start very publicly shipping a cryptographically secure hash, you better keep it cryptographically secure
So if my API happens to return text strings that always happen to have an even number of characters, I better make sure that all future versions of it also always return an even number of characters, just in case some moron decided to bank their application's functionality on that? No. If you decide to write a fragile application tethered to some incidental property of some upstream software, your application deserves to break.
Hey, I'm just the messenger here, if you don't like it, take it up with Hyrum ;)
But seriously: If you can afford to break your user's applications if they "deserve to be broken", sure. Many API maintainers can't, or at least don't want to.
In the latter case (which is honestly the norm rather than the exception, at least for public APIs), yes, you should better think about all implicit API contracts your API shape might be projecting. That guideline has served me very well through my career, at least.
> No. If you decide to write a fragile application tethered to some incidental property of some upstream software, your application deserves to break.
Just because you never made any promise regarding one aspect of your API doesn't mean that you're absolved from responsibility when you choose to change it. If you know for a fact that many users rely on it and you choose to break it, you need a good reason. That kind of balancing act is part of your job. If you don't respect your users, perhaps development wasn't the right career choice.
As developers we're of course constantly tempted to rename things that we named poorly, or change a schema that is no longer optimal. But we must always take a step back and think about the downstream impact.
> I better make sure that all future versions of it also always return an even number of characters, just in case some moron decided to bank their application's functionality on that?
Oh yes, this does happen. There's even a name for that: ossification (https://en.wikipedia.org/wiki/Protocol_ossification). You can't change your API/protocol, because "some moron" started depending on implementation details.
It's easy to say "your application deserves to break" from an ivory tower, but it's often not easy or viable to fix it (for instance, it might have different owners, it might no longer be maintained, it might be more expensive to change, etc). And the change which broke that application was not in it; the blame naturally goes to what was changed last.
Linus is one of the last remaining champions of rationality in large-scale software projects. Otherwise, it’s filled with devs who love to leave their brains out when it comes to practical scenarios. Anyone who thinks there is a security issue here is an absolute moron.
Rationality is the key point. A lot of devs approach their work in ways that are not rational or practical. Like the dev who wants to build a gold filigree decorated elevator that can handle ten thousand pounds of cargo and moves with the smoothness of a magnetic levitation rail in order to reach the second floor when all you really need is a ladder for the 2 people that will need access.
Thanks for writing this. I'd only been loosely following it and I hadn't realised how bad this is going to be. I have repos with tens of submodules and it's going to be a nightmare if any of them switch to sha256 in place. Not to mention I won't be able to use any new projects unless I rebuild my repo and all the submodules therein.
I thought the master to main thing was bad enough but this is going to suck. And just like the master rename it achieves basically nothing.
What is it about these projects that attracts people who just want to change things for the sake of it? Real engineering means coming up with a solution for backwards compatibility. This is just irresponsible and, frankly, a fuck you to everyone who will be affected by this.
« people who just want to change things for the sake of it » — Not for the sake of it, but to “make the world a better place.” The intention is noble (well, mostly, at least let's assume it is). Of course, there is disregard for history and her deplorables (or not so deplorables), comes with being “progressive”. Which is why I like Windows better (not 11 though, will probably have to go back to Linux at some point).
« What is it about these projects » — Maybe that they're “at the forefront.”
392 comments
[ 1.4 ms ] story [ 27.2 ms ] threadAs migrations go, it's reading as simple to me. You'll just have to backpoint the commit signatures. I must assume there's a backwards compatible reference for them in git 3, right?
Or drop them and reference the old structure in a dire pinch.
Once you rehash the entire repo, every single one of those external references will be broken. Because no, there's no support for looking up old hash -> new hash or the reverse.
- It's implemented in a non-backwards-compatible way
- The benefits over the older model are a bit nebulous
- There's a large amount of tooling that needs to catch up, and little sign that there is movement there
(Yes us Hacker News users have plenty of use cases for IPv6, like self-hosting and peer-to-peer networking and so on; we are not the average user.)
This effect doesn't exist for the Git migration. Each repo can be updated independently; it doesn't affect users of other repositories, and most likely, the majority of devs will work on some SHA-1 repos and some SHA-256 repos with no issue.
If anything, I would compare it with the Python 2 to Python 3 migration, which was also painful, but succeeded eventually (despite being much less necessary in the first place).
The GitHub Actions ecosystem found out the hard way, through some rather high-profile compromises. They hotfixed it by adding "immutable tags" to their platform, and are now working on adding a lockfile to... easily reference a commit hash.
repo can also rewrite existing commit and you again won't be able to retrieve it so switching to commit IDs only lowers the level of failure somewhat
> lowers the level of failure somewhat
congratulations on lack of ability to read with understanding
This seems weirdly aggressive and not nice.
Funnily enough, self hosting is why I can't use IPv6. I want vlan isolation, but only get a /64 from my ISP.
Fortunately the lack of IPv6 also isn't a meaningful loss anyway so whatever
You could put your server at x::1 and static-route that address as a /128 on your router, if it supports it. The reverse route might be a bit tricky but putting ::0 on the router and telling the server it's a /127 should work. Anything outside of the /127 (so, all the randomly generated addresses on your home network) would go back through the router.
Now if that /64 is also changing every day, then it's a problem and idk what you'd do.
Github is THE main platform for git. If github doesn't upgrade (and their code has been shit and hard to fix/update before) then the shift will not happen. Because yes, you can upgrade your repo independently, but if there is nowhere to push, no one will do it.
IPv6 is (also because of github) a great example for this. You can easily have an IPv6 address next to your IPv4 address, but a lot of websites (e.g. github) don't have that. Why would a normal company use IPv6 if even the bastion of nerds doesn't use it?
And to Python 2's "eventual migration" I can unhappily tell you, that my company (recently) bought an actively developed tool, that still uses Python 2.
One could wrap a whole merkle-tree with an additional extension tree, that just adds the new hashes. That way both kinds hashes could be used to traverse all data. The new hashes could be used to check the consistency, the old hashes would still be there to use in UIs or old release documentation. The downside being that you introduce more nodes in the overall data-structure which will have to be supported basically forever. And if SHA-256 is to week a third layer would need to be introduced. But the point is, it could be done. Albeit it would loose some of the elegance of the data structures involved.
All repos need to end up using SHA-2 exclusively by the end. All tools that speak only SHA-1 need to be made incompatible intentionally. If the SHA-1/SHA-2 hybrid approach could allow that to happen, then it would be useful. If not, then it would just be a waste of time.
This is very very easy to fix if you run into it.
1. Adopt git 3.0 if you can with sha256.
2. If you can't use sha256, set the config to put things back to sha1. Wherever you need to do this you probably already set dozens of ENV vars or settings, just add a new one.
Or write a 15 page analysis about how the above is so hard people will probably just find it catastrophic to even think about.
If there was an interruption to services on that night in particular, that's a serious public safety issue.
We had tested our system, and integration tested with all the other systems we connected to directly, but any FMECA analysis would show you that there were failure modes that we couldn't mitigate.
So people on planes falling out of the sky? Probably no.
People being crushed in a railway station on NYE? Possible yes.
If you read the OP article, the entire point he's making is that this would never happen, because a hash algorithm being "broken" doesn't matter in practice, because true supply chain security has nothing to do with file hashes.
SHA1-hashed objects should be able to refer to SHA-256-hashed objects, although this seems somewhat pointless.
But SHA-256-hashed objects should also be able to refer to SHA1-hashed objects, with a major caveat: if those objects themselves are part of a collision pair, then there is a genuine problem. But this is avoidable! Suppose that Linux decided to migrate to SHA-256. The upstream project could choose a pair of dates, say January 1 2027 and March 1 2027. Up to the first date, maintainers would be welcome to submit hashes of objects that are not yet in the repo but that they think they might submit later on, and, on that date, the upstream tree would finalize the list of these objects and reference it in the repo (with a new mechanism for this purpose). Effective the second date, the repo would start publishing SHA-256 commits and would never again accept a SHA1-hashed object that was not in the repo at the cutoff date or referenced as part of the Jan 1 block.
And now it would be impossible to get a new SHA1 collision in to the repo.
The only new git features needed would be:
a) actual compatibility so that a SHA-256-hashed object could reference a SHA1-hashed object
b) a new object type that's a list of allowed SHA1 hashes (or probably a tree of them) that is itself hashed with SHA-256 and a mechanism to link to one of these from a commit
c) a policy mechanism to set a repo to only allow SHA1-hashed-objects that a reachable from a preconfigured SHA-256-hashed commit
In any case, even if Git 3.0 were completely incompatible, it would suck, but it's not the end of the world. You just treat it as if you were migrating from one SCM system to another. CVS -> SVN -> Perforce -> Git -> Git 3.0 -> [...] been-there-done-that. This is something that both open-source and commercial projects have had to deal with over the years.
Or maybe it would be a repeat of Python 2.x -> 3.x. ¯\_(ツ)_/¯ With AI assistance, hopefully porting the tooling over may go a lot quicker and smoother.
[0] https://www.youtube.com/watch?v=eJJp0RE7cd4&t=1134s
I don't see why all users wouldn't want to keep both hashes.
I don't know what the solution is, but I'm inclined to believe that any repo with a single SHA1 commit is as weak as a repo with all SHA1 commits.
* I host a mirror of the Linux git repo.
* You download Linux from my mirror.
* You check out a commit, say fd179f8a05be3ccae366b9b96e176b51fbe54aab, which you know is a genuine commit through some out-of-band mechanism (mailing list, GitHub web interface, a line in a Nix file, whatever).
* You check whether the repository I gave you is legitimate or not by re-computing the hash of the commit which I claimed was fd179f8a05be3ccae366b9b96e176b51fbe54aab. If it comes out to be fd179f8a05be3ccae366b9b96e176b51fbe54aab, you know it's legitimate. If it doesn't, you know it's fake.
This is a completely normal use of Git. People download from mirrors all the time. People rely on commit hashes to identify a specific source tree. People trust that if whatever the mirror gave them hashes to the right value, it's genuine. That way, you don't have to trust the mirror.
If I can forge my own commits to have any hash I want, this whole model breaks down. I can replace some old commit in the repo with my own forged commit with the same hash, and when you download a copy of the Linux repo from my mirror, you'll receive a repo with malicious content, but it'll hash to the same fd179f8a05be3ccae366b9b96e176b51fbe54aab hash as a genuine repo would. This breaks the security model of Git.
That's a 160 bit hash, which is SHA-1, which has the security properties of SHA-1.
Suppose you check out a commit with a given SHA-256 hash. That commit object represent the root of a tree where all the edges are hashes (and types, etc). I'm suggesting one of two designs:
a) (Simpler but weaker) If Linus has published that commit, then he is confident that he hasn't pulled in any too-new SHA-1 hashes and that there are no collisions present in what he thinks the tree is. So, by induction on the traversal depth, there is only one actual object identified by each edge, and those objects contain the hashes of their child edges, so those hashes are all correct.
This breaks if there is a malicious collision already in the tree.
b) (Stronger but higher overhead and more complex) There would be an object or objects, discoverable from the root by following only SHA-256 edges, that encode a duplicate-free mapping from SHA-1 hash to SHA-256 hash. The client finds and parses that and then, as it traverses the tree, each time it reads a SHA-1 hash, it computes the SHA-1 and SHA-256 hash of the referenced object, verifies that the pair is in the mapping and also verifies that the SHA-1 hash matches what the edge requires.
I think that (b) is genuinely cryptographically secure in the sense that, if you can construct a commit that has the same SHA-256 hash as an official upstream commit but different contents, then there is necessarily a SHA-256 collision.
For B), I would think this could work, but it's a completely different solution from what you proposed and what I responded to.
How? Remember, there are (currently, anyway) no known SHA-1 preimage attacks.
> If I can forge commits with any SHA1 hash at will
We probably don't want to wait until there are practical pre-image attacks discovered to change away from SHA-1.
I think I stand by my second proposal. I also think it's absurd that, after all these years, upstream git still can't figure out a credible migration plan.
More importantly we just shouldn't use your mirror if we don't trust you. If you're evil you're probably lying about all the tags and branch tips anyway.
The "commit before" might be compromised, but the git commits refer a snapshot of a tree + a list of previous commit IDs, so the "new" SHA256 commit will not have any files altered
Presuming there’s some validation that SHA-1 names are unique, then that should be safe — the only way I can see one could do a pre-image attack is either fetching from a SHA-1 server (because then you don’t get the SHA-256 object name), which requires a second pre-image attack on SHA-1 (known to be feasible); or by having a second pre-image attack against both SHA-1 and SHA-256 simultaneously (and SHA-256 is still believed to be secure).
https://lore.kernel.org/git/Pine.LNX.4.58.0504291221250.1890...
As linked by another commenter in this thread, Linus worked out years ago that even if someone inserted a malicious object into the kernel repo, it would at best be a nuisance and not a major concern.
> We could be using MD5 and it would honestly probably be just fine.
This will be a train wreck. I hope they don't release before adding compatibility modes to keep the existing sha1's around in the database.
[1]: https://github.com/newren/git-filter-repo
[2]: https://htmlpreview.github.io/?https://github.com/newren/git...
I don't think this is going to be a problem at all.
Not talking about archived code, but active projects that pre-date git 3.0
That way an old commit with a message having a sha1 can reference the archived version, and new commits reference the active version.
For massive perf and mem use damage. But oh well. And then we will wait for official version
One example is to maintain git-replace refs for the rewritten SHAs but there also exist config flags to enable object format compatibility extensions that help translate the SHAs back and forth.
https://git-scm.com/docs/git-replace
So this is the right time to post that everything they’re doing is wrong? Did you engage in all the discussions about it and how best to handle it? Whether SHA-256 was the best solution?
I don’t see anywhere that it talks about alternate proposals or why they might have been better. Why the particular suggestions here were rejected.
This seems like a bunch of Monday morning quarterbacking.
I used to be one of the more vocal defenders of C++ but honestly, if the reason you’re defending git submodules is because there’s no better option in C++; then you’ve basically already lost the argument.
Literally every other language has found a better alternative. Even Go has, and that’s largely mocked for its prehistoric approach to modern programming language design.
thought "costly" in the title and "incomprehensibly expensive" in the subheader meant this piece would discuss how much less performant sha-256 is on modern machines, but didn't see anything. isn't there hardware acceleration? how much worse is it?
I just sent a patch series to the list that enables sha1dc to be accelerated on modern CPU architectures to close to normal SHA1 speeds, but since it was ported from a Rust project by an agent, it will never be applied.
https://lore.kernel.org/git/20260929112544.86511-1-scott@git...
SHA-512 is always faster in software than SHA-256, when run on 64-bit CPUs, and it is also faster in the CPUs that support both SHA-256 and SHA-512 in hardware.
Arm-based CPUs have supported SHA-512 already for many years and the latest Intel CPUs also support it, i.e. Lunar Lake, Arrow Lake S (S is for desktops, Arrow Lake H for laptops does not support it), Panther Lake and Clearwater Forest.
I expect that AMD Zen 6 should also support it, because they are the last important vendor without SHA-512 support.
All modern CPUs support SHA-256 in hardware, so it is faster when SHA-512 is not supported in hardware, otherwise SHA-512/256 is preferable, by being both faster and more secure.
1) It's claiming SHA1 insecurity is theoretical, while SHAttered from 2017 was specifically a pratical proof of concept. The only reason Git wasn't affected, is because they didn't bother bruteforcing a git-blob prefix.
2) It's claiming collision attacks don't matter, only second-preimage attacks do. This is incorrect, collision attacks are enough for code-smuggling problems, when two repositories are on the same git commit (verified by the full commit hash), yet contain different code in their git checkout.
3) The Linus quote "The real security is in distribution" is arguing that "git's content-addressed system should not be used to address content". It's arguing that, in case of curl|sh, you shouldn't use a sha256sum-gate to pin the content to something you've reviewed, you should instead ensure curl is fetching from an https server.
2) I specifically argue that even if both attacks were practical and cheap, it's still not the problem we should be focusing on.
3) Have you read this email (that I linked to)? It is almost the same general message (20 years ago) that this blog post is. It literally goes though a theoretical object replacement attack and how dumb this scenario is and so SHA-1 is fine.
https://lore.kernel.org/git/Pine.LNX.4.58.0504291221250.1890...
It seems unlikely it will stay that way forever. Typically attacks get more efficient over time as researchers find improvements, not to mention computers getting better.
In 2015 it was estimated to cost $100,000, now the estimate is down to $10,000. Where will it be in 2035?
Then you would have security researchers making conflicting claims depending on which repository they first pulled from, even though they are on the same git commit hash.
People assume sha1 git is cryptographically sound, and a git commit is a secure identifier to reason about source code, whether you and me like it or not.
Still, eliminating the risk is a good idea.
... for Linux
... and developers working for it constantly
the attack wouldn't work. Joe Schmoe? It's worse than just "being compromised"
You have repo of dependency locally, let's assume you downloaded good copy, the commits get compromised, you're safe.... right ?
Nope, if there is build server along the way and ESPECIALLY if it practices building from clean state every time, the build might be infected while your local copy is clean, giving no chance to notice it, unless your entire chain including local builds are reproductible AND you actually check it
Conveniently Tom didn’t mention anything about Edward Snowden and what he published. That was basically start of TLS everywhere.
Then he didn’t mention ISP idiots that were actually injecting ads to cute websites like Tom’s. I hope Tom likes when his website is used by ISP to make money on ads he doesn’t have any control over.
Then he goes on to criticize certificate transparency, but it works. Companies got kicked out from trusted root program because they were doing stupid stuff like making certs they shouldn’t.
Let’s not forget glorious state of Kazakhstan where without TLS they would just listen to all traffic - well with TLS they were trying to pull MITM but were uncovered and got their stuff removed by TLS ecosystem.
While he does indeed have extensive knowledge of TLS/SSL. He still completely side steps points I wrote about and exaggerated many minor inconveniences. While PDF seems quite up to date it also picks on stuff that is not there anymore like green padlocks.
ISP injecting ads is like annoying but if someone knows as much as Tom about TLS and totally skips rouge "airport wi-fi" can use his website to own someones else device that is the argument I should use for calling him or anyone else names on the internet.
I guess Kazakhstan example kind of covers it, because I do believe they would definetly deliver malicious payloads to dissidents.
You mean like how Google makes money showing ads against your content that you don't control? You mean how basically every ad network works?
I could just as easily argue that by paying my ISP and signing up to their T&Cs, I've opted in to seeing their ads, not the ads from some random website.
With Google you have to make an agreement and put piece of code in your website willingly and then you get minimal cut but still, that is totally your choice.
Impractical for an individual, definitely. For a large org, maybe, but if the payoff was big enough? For a nation state level actor intent on doing something, absolutely not.
The go-to example is Stuxnet. Some countries wanted to attack Iran's nuclear enrichment programme, so they spent 5 years developing a worm that used multiple zero day exploits to attack a specific controller in a specific model of gas centrifuge. Could Mythos write Stuxnet? Unlikely, but a knowledgable team with access to it could probably write it in a lot less than 5 years.
'impractical' has very different values for different groups.
I can see that some things might have a risk profile that might possibly make all this costs still worth it, but does it make sense to have these unicorn projects effectively blow up 20 years of ecosystem?
Shouldn't the extra cost of doing something out of the ordinary be carried by whoever does something out of the ordinary?
This feels like a bridge to be crossed when one gets there (if at all).
__
FWIW, we actually do have a choice here. No one is forcing the industry at large to adopt an unpatched git 3.0 binary built from a source that makes that a default.
This should be a trivial overlay to carry around with effectively no downsides. So convincing whoever is steering that ship doesn't necessarily matter, as long as enough sane pragmatics agree on how defaults should actually be.
The rationale of mass migration is that if you don't impose it, nobody migrates. This has notably been the case with famously insecure SSL parameters (512 bits RSA keys, PKCSv1.5...). And many companies may believe they are not critical, which might be true until it is not.
Case in point: you manufacture walkie talkies and suddenly your products have bombs inside. Or you maintain a compression library for free and suddenly you are shipping a backdoor to all Linux products.
It instead questioned to which degree execution of them is reasonable in a world that does not contain infinite resources.
Everything is a trade-off. Not all of them make sense.
In order to compromise the big player, you only have to compromise the weakest link in its supply chain. In effect that means that leaving the migration optional is as useless as doing nothing.
It is not of my concern to live in ways that are harder for me, just so that big tech can have it easier.
Jesus man.
Feel free to tell that to journalists investigating corruption ...
> I assume that what you are doing on your free time is not worth governmental attention.
Definitely does not apply to journalists investigating corruption.
> Definitely does not apply to journalists investigating corruption.
And that bring to mind another aspect - When strong security is not the default, anyone using it looks suspicious in some eyes.
https://www.change.org/p/stop-the-chat-control-european-regu... is worth a read, IMO!
As for "nobody will reasonably use a quantum computer to snoop on your home IoT daily reports", they already have access to your data one way or another, so really no need for a quantum computer. :P
I don't remember where the compromise happened (factory, distribution, sell point), but very clearly at least one of them did not expect to be a critical asset in the Israel - Lebanon war.
> And many companies may believe they are not critical, which might be true until it is not.
I'm at a K-12 public school. That shouldn't be on the front lines of a war with Iran, but, in cybersecurity terms, we are. If you disrupt a school district, you disrupt one of the largest employers in the area. You also disrupt the largest childcare facility in the area. The amount of economic damage you could inflict on a community by disrupting the public school system is pretty extreme compared to the amount of funding provided to protect it.
> Or you maintain a compression library for free and suddenly you are shipping a backdoor to all Linux products.
You already showed yourself that your naive assumption is wrong for a relevant group that uses Git.
Assets are non critical until they become critical.
What does that mean?
Linus's "what matters is distribution" comment also doesn't make sense when merge effectively is distribute. Which, again, is the reality of supply chain.
Stuxnet is essentially "boutique malware". You can buy it/have it built with enough money/resources.
Weaponizing a cryptographic algorithm with some theoretical vulnerabilities (but no by-design backdoor built-in) is a totally different game. And TFA is right that it's a dumb endeavour. You can probably "stuxnet" your way in for much cheaper.
In all the years since 2017, with all the orgs having huge GPU-filled data centers (and’s an interest in software security) has anyone demonstrated a real git collision?
there's a lot you can do behind the scenes with even a couple bits of... free space?
the key should be enough of a unique id to not have to require this? see (hash). a separate linked 'randomized' unique identifier might be crazy man territory but i'm not lying... your 'weakened' key doesn't _need_ to be all zeros, just predictable eg. within a certain time frame, anything that can reduce the search space is dangerous.
edit: it could contain an identifier for which hrng was used to produce it.
At some point "The world is actually ball shaped" just makes a lot more sense than the thousands of years of vast elaborate conspiracies to keep you from realising that there are Mole People whose underground civilisation is accessible from Antarctica. So in the hopes that it's the former (you just didn't understand), I shall endeavour to explain.
The MD-series and SHA-1 and SHA-2 series of cryptographic checksums use what is called Merkle–Damgård construction which operates on fixed sized blocks of data. In this design if we can find a collision before a certain block, everything after that point will keep colliding. The converse doesn't work, there aren't suffix collisions which would work for any prefix, only prefix collisions which work for any suffix. In SHA-3 and newer hashes a "Sponge" construction is used, we just pour stuff into the sponge and only squeeze out a fixed-size hash at the end, so these "prefix" attacks would need to change the entire state of that sponge.
An X.509 certificate's serial number is very, very early, before any of the information which can be chosen by the recipient. So by ensuring this number is entirely random we're making it impossible to construct information which results in a collision, the prefix for their input will be random so it's now impossible.
This is a "defence in depth" strategy. The SHA-256 hashes used are believed to be fine, for the immediate future, but even if they were vulnerable to a prefix attack as we know SHA-1 is, the choice to have random serial numbers defends us anyway, the attack wouldn't work on certificates.
Asking to trust in an authority (while the main authority Microsoft/GitHub has is essentially figuring out enterprise sales well enough to be acquired by a company desperately needing developers after fumbling badly in the 2000s) is exactly the opposite of my stance - it is a large corporation, with heavy employee rotation, with substantial exposure to various forms of regulatory pressure and to various forms of corruption.
Which is why the cryptography exists to prove the developer-to-consumer trust without trusting the intermediaries. Yes, I do check GPG signatures. Yes, I do include a git commit hash in the binaries I build. And surely I want to make sure that this doesn’t mutate because some unknown engineer at GitHub had a bad case of gambling debt.
Probably a naive question, but why not kill two birds with one stone if it can be done for a reasonable cost?
When I check out code from a git repository in a pipeline using a git hash, I expect the code to be exactly what has been reviewed by me under that hash.
Everything else would just be a crazy invitation to make supply chain attacks uncircumventable.
Turns out securing a service to transfer a single hash is a lot easier than securing a service to transfer gigabytes of data.
Even if I don't fully trust Github, it is still incredibly convenient to be able to upload my code there and then send someone an email telling them to fetch commit `123abc` from some repo link. As long as my email isn't compromised, that should be secure.
I pushed aviation toward starting with SHA-256 instead of SHA-1 for code signing more than fifteen years ago, just after NIST first started discouraging the use of SHA-1 in new code.
But you still need SHA256 for that
Or if first writer wins, and I know that you have a popular non-GitHub repo that you're about to migrate into it, then I could pre-poison the namespace by writing my own version of a commit that I see you already have in Codeberg or Savannah or wherever.
I don't swear that this is how GitHub actually works, but I've had knowledgeable friends swear up and down that it is. And honestly, it'd make sense. They could shard storage by the first 4 digits of the hash or something, and that'd be vastly more efficient if all commits were writing to the same space.
But the very wrong assumption here is: "if I see that you pushed commit abcd134, and then I can build and push a colliding commit, and the backend doesn't check uniqueness before writes"
All parts of this are incorrect.
You can push a colliding commit to a fork, but Git will see that it's already there and ignore it - first write does win. Also, "backend doesn't check uniqueness" is also wrong. The server will check for collisions and if this particular case happens, the server will see this and warn you _AND_ not write the object.
All the details on how GitHub's infrastructure has evolved over the years are very publicly detailed in the GitHub engineering blog and in technical talks. There are no "secrets" or "rumors" here, all the information is one google search away. Perhaps you need to re-evaluate your priors on how knowledgeable your friends are.
- https://github.blog/engineering/architecture-optimization/in... - https://www.youtube.com/watch?v=Ri8hSZNKzu4 - https://github.blog/engineering/building-resilience-in-spoke... - https://github.blog/open-source/git/counting-objects/ - https://www.youtube.com/watch?v=DY0yNRNkYb0 - https://cursor.com/blog/git-at-any-scale
You cannot do that. This is called a second-preimage attack, which has not been demonstrated even against MD5, let alone SHA1.
Yes, they ought to be good enough. That has always been git's security model.
Note that the commit doesn't have to be communicated over the same channel as the git data.
> Well, what about someone who is fetching the commit from that server for the first time and has nothing to compare the hash against?
Then they're vulnerable. But what about somebody learning about the trusted hash in another way, e.g. a build server getting an internal call authenticated by an authorized developer?
Just because you can think of an insecure way to use git hashes doesn't mean there aren't any other, secure ones.
Yes, but you seem to be saying they are not, or maybe just that git hashes are somehow inherently not capable of taking that role, unless I'm misunderstanding.
Wait, why is anyone expecting that to be a good idea at all?
How does this matter? When I have a machine that I trust and a git hash that I trust, I don't need to rely on the transport mechanism. As long as the content cannot be forged to match the hash, the transport is completely irrelevant.
And if git provides a cryptographic hash over the content, I don't see why it shouldn't be used to verify the integrity of the checked-out content.
Sorry, no.
Again, hashes are by definition, insecure. I.e. you don't have a guarantee that the hash references the same commit, just a (very, very strong) probability that it does.
> Everything else would just be a crazy invitation to make supply chain attacks uncircumventable.
What? Uncircumventable? Logically equivalent, I read your statement as "So if all cars are not blue then they must be red"? This does not follow...
I like the last part of the article, which proposes a (very reasonable sounding) extension for people who care (more) about their code-sec. You should be able to swap out your hash algo without having to rebuild your content addressing system.
Separation of Concerns, people...
This doesn't seem to be a useful definition. Would you classify every computable algorithm as insecure, because by generating a random bitstring, there is a (very, very low) probability of guessing the hash/secret key/solution/signature?
Cool, you've just defined the foundation of signatures and web encryption as insecure. What next?
Commit signing indicates otherwise.
We use SHA-1 in our storage mechanism, which predates git. There is a (quite long) written threat model. Someone being able to generate collisions with an enormous amount of effort under just the right conditions is not a threat under that model.
There are a lot of tools where the lines blur between “for the users” and “for the development team” because the users benefit from some things that make the developers’ lives easier.
They absolutely are, both when you're using signed commits (i.e. GPG, SSH, and S/MIME) or just identifying a given commit/repository state by hash via a secure channel and then serving object data over untrusted transports.
It's entirely possible that you don't use either, but that's certainly not true for everybody.
Calling anyone using either to be "doing it wrong" is borderline gaslighting: Git used to have these security guarantees, and just because they're now broken doesn't mean they were never there in the first place, or that it was stupid to rely on them.
That you are wrong. Many people and orgs rely on this.
Does not matter if you think it's stupid, for them it is. live with it.
Package managers even use it. E.g. you can have a cargo dependency pointing to GitHub at a specific commit. It's definitely intended to provide end to end security without depending on GitHub being secure.
Also git submodules.
Absolutely the SHA-1 is treated as "authenticating". Cargo.lock (for regular crates.io dependencies) are confirmed using SHA-256.
And we are talking about who knows how many tools that work with git built in the years, and this is also made it worse from the fact that most tools just invoke the git binary and capture its output instead of passing from a library.
I like more the solution proposed at the end of the article, do not change sha-1 but instead, if you are relying on git commit for security purposes (that was never the intended use) add another header to the git object with a sha-256, so that with the small expense of computing the hash twice you don't break 20 years of existing tools that make the assumption of the git commit being 40 character long.
git doesn't change that.
everyone is already using sha256 everywhere. i am. sha1 is only still around because github forces it for pretty urls
An advantage of a hash with a different length is that a full length sha1 commit hash and a full-length sha256 commit hash can't be confused for each other
I don't simply mean to be disparaging - its important to security that the people making the decisions are a) competent b) can read, otherwise any "security" decisions they are making are at best probably insecure, and at worst, causing active harm and insecurity, DOS, etc...
Generally, I think you didn't read (or at least comprehend) the article:
1) Your assertion is factually incorrect. Practical proof of concepts are not "CVEs exploited in the wild". The article is not claiming that SHAttered is not correct, it in fact references it.
2) I don't think you read the article. The article claims the exact opposite, and in fact addresses the issue WRT to distribution.
3) Your sentence here is very confusing - I'm going to give you the benefit of the doubt and presume that you mean to say that the problem here is in the security of the naming. HTTPS itself has nothing to do with distribution security, that would be DNS/SecDNS (IFF you are using git+http protocol, then HTTPS is relevant, but not to git otherwise). But this is exactly what the article was talking about, the distribution is the security, not the hashing algorithm.
Submodules is a legitimate argument against this, though I don't know how widely this feature is actually used, and similar to the arguments in favor of switching the default branch from master to main, this is simply a setting which can be changed.
I do like the idea of commits having both hashes, and am surprised that idea has not been explored further.
Generally though, I think the author's strongest argument is simply that the change isn't strictly "needed", and all the other issues presented aren't the strongest arguments against change.
SHA-256 is considered quantum safe by the NIST and is left out of PQC migration guidance entirely.
Things like this have a tendency to to be revised as time passes.
We have a content addressable storage system internally and we use a self describing hash for it, so we can change the hashing scheme for different use cases.
Multihash is ipfs’ container for this.
Best I can tell, all a forced collision would do is let someone who already has control of a repo modify the history in a far from plausibly deniable way. Which in practical terms, they already could do simply by replacing the whole thing, because who's out here using git hashes as a security tool? Every pinning I've ever seen has been to tags (which can be modified at will), or hashes of the actual payload (which doesn't need to be the same as what git uses).
I would write this to the mailing list, but I thought a conversation that includes people outside that list is more interesting to me. Ultimately I'm not sure if I'm dumb about this or the whistle blower that's willing to actually say "maybe this isn't the right call"
If you're replacing the weakness of SHA-1 just by going to another algorithm, you better be prepared to go to the next one when sha256 collisions happen, and it doesn't sound like git's design would be easy to modify for this type of crypto agility.
I do like their proposal for using signatures to establish trust and allow swapping sha256 for whatever comes next.
However, it's not a git problem. It's an ecosystem problem. It's that every git repo has to choose one and they're entirely incompatible with each other. That is the cost and the difficulty.
It took 20 years to go from vulnerability in sha-1 to having to replace it out of caution. There is no such vuln in sha-256 yet. It could easily be 25 years before we find one, and another 25 years before we have to do something about it. Will git still be used 50 years from now?
All it takes is just one collision to consider it broken right?
But hey maybe the attempt to fix it makes git controversial enough it falls out of favor, and nobody uses it anymore in 2 years, problem solved? sure.
No, its considered broken before that stage. i.e. when someone discovers an attack that would allow someone to create a collision faster than they should while still being impractical.
> With the kind of compute power available nowadays and AI models I wouldn't be surprised we see it much sooner.
Computer power doesn't super matter, what matters is algorithmic breakthroughs. So far i dont think there are any examples of major breakthroughs of that type via AI, although perhaps i am just misinformed. Its still early in the AI revolution, it might still happen, but as it stands i don't think there is any reason to worry about that.
"Both Fossil and Git started out using only SHA1 hashes. But when the SHAttered attack against SHA1 was published on 2017-02-23, the need to migrate to a stronger hash algorithm was recognized. Fossil added the ability to use SHA3-256 as an alternative on 2017-03-01 (six days after the SHAttered attack was first published). SHA3-256 is now the default for all new repositories and check-ins in Fossil, though older check-ins that occurred prior to SHAttered can still use their original SHA1 hash. Hence, no repositories had to be rebuilt and no hyperlinks were broken."
https://fossil-scm.org/home/doc/trunk/www/hundredandone.md
To me it's so interesting watching in realtime Git is still battling with this decision and for Fossil it was just another week of development.
That whole page is fun to read. Another fun fact somewhere else in the docs is that Fossil uses a grow-only set to store check-ins. They came up with this scheme some years before it was formalized by CRDTs!
Fossil isn't difficult to change not because it's technically harder for Git but because Git has a community and ecosystem that Fossil does not. The cost is not in the individual project for Git, the cost is because there is _so much_ in Git and this bifurcates everything.
(… looking at the parent, though, I imagine there might be some information from the inside…)
That's is, since only recently, no longer strictly true: the age-old libfossil recently got client sync support, so is now (aside from _serving_ repos) essentially a standalone impl (its own developer still uses fossil(1) stash, patch, and diff -tk features, but otherwise uses libfossil's counterparts).
Also, Dan Mestas has <https://github.com/danmestas/go-libfossil>, a Go library for working and fossil, and he is also working on <https://zeitforge.app/>, a clean-room impl. of fossil (whereas libfossil is largely ported directly from fossil(1)) which even goes so far as to _not_ use an sqlite database for its file storage.
Dan Mestas and Dan Shearer are working on finalizing RFCs for fossil's sync protocol and artifact format, and zeitforge is created by carefully managing LLMs which are reading that draft (but not the source code of libfossil or fossil).
:%s/git 3\.0/python 3.0/g
:x
Neovim's lazyvim plugin sucks because it takes over H C and L.
https://support.microsoft.com/en-us/windows/deployment/updat...
> Windows 10 support has ended on October 14, 2025
You don't think that if we had everyone on the same Windows version we could streamline things enormously?
For MS, for developers, for hardware manufacturers, and so on to be able to deploy, for example, ipv6 since everyone is on the same stack? Same stack, same bugs, same fixes.
The difference in effort between updating one versus two very similar code bases is minor, especially when "Windows 10" and "Windows 11" each refer to multiple versions already.
For drivers you only need one version to support both.
IPv6 has been built into Windows for ages. If you think they can force toggle it or something, that won't work at all and deleting Windows 10 wouldn't make it easier.
I can't think of a situation where consolidating Windows versions makes a big difference. I can think of useful situations based on getting rid of all other OSes and/or Microsoft instantly forcing out updates to everyone, but those are different scenarios and they come with severe downsides.
How about testing? Do you think that there's no testing put out when a fix has to go out for 2 OS, regardless of how similar they are?
> What problems are solved by getting all Windows users onto Windows 11 in particular?
I have given a few. If you don't want to see it, that's fine.
I would estimate that supporting both windows 10 and 11 is a single digit percentage harder than supporting just one of them.
> I have given a few. If you don't want to see it, that's fine.
Where? You said something vague about streamlining, and the only concrete example was something about "deploy ipv6" which Microsoft did back in Vista.
(Because: This very old problem is ongoing with git, and is closed with Fossil.)
This problem isn't hard or new. Just look at things like a TLS handshake. You need to separate the protocol from the storage implementation.
I personally believe the initial Git programmers were too in love with the efficiency of doing a bitwise 160 bit comparison on the stack and they sacrificed the known issue of changing the algorithm to do it. A decade earlier the same thing had happened with MD5.
1. Collisions aren't as bad as preimage attacks
2. Even if you made a file-with-malicious-hash, how would you get people to pull it?
3. Other attacks are a bigger problem (social engineering)
(2) is laughable in a world with github. It's common for unknown people to submit pull requests to code bases, and for those changes to be reviewed and merged. For example, as part of reviewing pull requests, I have `git fetch`'d proposed changes to my local machine to check behavior on some additional test cases. "If you fetch it you're fucked" is unacceptable as a security boundary.
(1) and (3) are just tu-quoque arguments about other attacks being worse. The relevant question isn't how bad other attacks are, it's how bad this attack is.
The fundamental problem with collisions is that software often assumes they can't happen (or is not tested against them). Thus collisions can trigger bugs, or otherwise cause surprising behavior. For example, webkit figured the colliding PDFs demonstrating a sha1 collision would be excellent for unit tests, so they merged the PDFs into their SVN repo... which completely fucked it [1]. I don't know the exact internals of git so I can't comment on how you would get surprising things to happen, but "oops the file you merged was different than the file you reviewed" and "oops the repository got corrupted" seem entirely plausible.
[1]: https://www.reddit.com/r/programming/comments/5vyhy2/webkit_...
(1/3 counter) is not what I argued. I argued from the worst-case position that collision and preimages were theoretically cheap and fast. Even in that case, I feel my arguments hold.
The main issue here is that you assume you can replace an existing object with a replaced one, which you cannot. Not only that, but in all known cases, the sha1dc variant of SHA1 that Git uses will even _tell_ you that someone tried to do this, which singles out the source quickly.
> but the point is the SHA-1, as far as Git is concerned, isn't even a security feature. It's purely a consistency check. The security parts are elsewhere, so a lot of people assume that since Git uses SHA-1 and SHA-1 is used for cryptographically secure stuff, they think that, Okay, it's a huge security feature. It has nothing at all to do with security, it's just the best hash you can get. ... [1]
[1] https://www.youtube.com/watch?v=4XpnKHJAok8&t=56m20s
So Torvalds used SHA-1 purely because he needed a hash function with no other property than identifying content.
Flexibility made sense in 1995 when nobody was sure which algorithms would stand the test of time. Even in 2005 it was unnecessary and in 2015 it was an outright liability.
Sorry if the video answers this, but how does commit signing work if it doesn't rely on the hashes obeying some cryptographic properties?
For all practical purpose SHA-1 is a bad hash function, it's slow, it's insecure.
If SHA-1 is not a security measure, why do you even sign the commit. It doesn't make any sense. You give a strong signature on something weak.
At the very latest, this fact was cemented when first-party git commit signatures started depending on the security properties of SHA-1.
So if my API happens to return text strings that always happen to have an even number of characters, I better make sure that all future versions of it also always return an even number of characters, just in case some moron decided to bank their application's functionality on that? No. If you decide to write a fragile application tethered to some incidental property of some upstream software, your application deserves to break.
But seriously: If you can afford to break your user's applications if they "deserve to be broken", sure. Many API maintainers can't, or at least don't want to.
In the latter case (which is honestly the norm rather than the exception, at least for public APIs), yes, you should better think about all implicit API contracts your API shape might be projecting. That guideline has served me very well through my career, at least.
Just because you never made any promise regarding one aspect of your API doesn't mean that you're absolved from responsibility when you choose to change it. If you know for a fact that many users rely on it and you choose to break it, you need a good reason. That kind of balancing act is part of your job. If you don't respect your users, perhaps development wasn't the right career choice.
As developers we're of course constantly tempted to rename things that we named poorly, or change a schema that is no longer optimal. But we must always take a step back and think about the downstream impact.
Oh yes, this does happen. There's even a name for that: ossification (https://en.wikipedia.org/wiki/Protocol_ossification). You can't change your API/protocol, because "some moron" started depending on implementation details.
It's easy to say "your application deserves to break" from an ivory tower, but it's often not easy or viable to fix it (for instance, it might have different owners, it might no longer be maintained, it might be more expensive to change, etc). And the change which broke that application was not in it; the blame naturally goes to what was changed last.
I thought the master to main thing was bad enough but this is going to suck. And just like the master rename it achieves basically nothing.
What is it about these projects that attracts people who just want to change things for the sake of it? Real engineering means coming up with a solution for backwards compatibility. This is just irresponsible and, frankly, a fuck you to everyone who will be affected by this.
« What is it about these projects » — Maybe that they're “at the forefront.”