Author here. This is correct, we set out to do binary diffing but we soon discovered that if you put similar enough object files together in a stream, and then compress the stream, zstandard does a fantastic job at…
Yes, this is less great than I would like! :( :)
Author here, I'd like to see such a comparison too actually, but I'm not in the position to do the work at the moment. We did some preliminary experiments at the beginning, but a lot changed over the course of the…
Yes you could do that. On the other hand, all revisions for a month is 100MiB, and all revisions we've built spanning 2019-now are a total of 2.8GiB, so we opted to forego implementing any object negotiation and just…
Nice, I bet dwarfs would do well at our use case too. Thanks for sharing.
ThinLTO can be pretty quick if you have enough cores, it might work. Not sure how well the LTO objects compress against each other when you have small changes to them. It might work reasonably. manyclangs is optimized…
> maybe mention this as well in the README? We've tweaked the readme, I hope it's clearer. It would be great to provide this for gcc too. The project is new and we've just started out. I know less about gcc's build…
(Copying from Q&A) Before starting out some time ago, I did some experiments with bup. I had a good experience with bup and high expectations for it. However, I found that quite a lot of performance was left on the…
<3
We've just added an applicability section, which explains a bit more what we do. We don't have any ELF specific heuristics [0]. https://github.com/elfshaker/elfshaker#applicability In summary, for manyclangs, we compile…
Author here. Compressed data is unlikely to work well in general, unless it never changes.
Author here. No architecture specific processing currently. Most of the magic happens in zstandard (hat tip to this amazing project). Please see our new applicability section which explains the result in a bit more…
Done, hopefully this is clearer. Please let us know if you see a way to improve it further: https://github.com/elfshaker/elfshaker/pull/60
(Disclosure: I work for Arm, opinions are my own) Author here. elfshaker itself does not have a dependency on any architecture to our knowledge. We support the architectures we have use of. Contributions to add missing…
Author here. The executables shipped in manyclangs are release builds! The catch is that manyclangs stores object files pre-link. Executables are materialized by relinking after they are extracted with elfshaker. The…
Author here. I've used bup, and elfshaker was partially inspired by it! It's great. However, during initial experiments on this project I found bup to be slow, taking quite a long time to snapshot and extract. I think…
Author here, I agree with xdfgh1112, please take care before using brand new software to store your backups!
Author here, this software is young, please don't use it for backups! But also, in general, it might not work well for your use case, and our use case is niche. Please give it a try before making assumptions about any…
Performing poorly with non-textual data happens for a a number of reasons. Binary data, when changed, often have a lot of 'non-local' changes in them. For example, a PSD file might well have a compression algorithm…
Author here, I don't think it would apply well to that scenario. elfshaker is good for manyclangs where we ship 2,000 revisions in one file (pack), so the cost of individual revision is amortized. If one build of…
Author here, it works particularly well for our presented use case because it has these properties: * There are many files, * Most of them don't change very often, * When they do change, the deltas of the binaries are…
Author here, I don't currently know how this compares to git-lfs. It it is possible git-lfs would perform quite well on the same inputs as elfshaker works on. If git-lfs does already work well for your use case I'd…
Author here. I'm unsure whether this would apply very well to cargo or not. If it has lots of pre-link object files, then maybe.
Author here. Maybe, it's a fun idea. I have toyed with providing a fuse filesystem for access to a pack but my time for completing this is limited at the moment.
Author here. We'd love this to be a thing, but this is young software, so we don't recommend relying on this as a single way of doing a backup for now. Bear in mind that our main use case is for things that you can…
Author here. This is correct, we set out to do binary diffing but we soon discovered that if you put similar enough object files together in a stream, and then compress the stream, zstandard does a fantastic job at…
Yes, this is less great than I would like! :( :)
Author here, I'd like to see such a comparison too actually, but I'm not in the position to do the work at the moment. We did some preliminary experiments at the beginning, but a lot changed over the course of the…
Yes you could do that. On the other hand, all revisions for a month is 100MiB, and all revisions we've built spanning 2019-now are a total of 2.8GiB, so we opted to forego implementing any object negotiation and just…
Nice, I bet dwarfs would do well at our use case too. Thanks for sharing.
ThinLTO can be pretty quick if you have enough cores, it might work. Not sure how well the LTO objects compress against each other when you have small changes to them. It might work reasonably. manyclangs is optimized…
> maybe mention this as well in the README? We've tweaked the readme, I hope it's clearer. It would be great to provide this for gcc too. The project is new and we've just started out. I know less about gcc's build…
(Copying from Q&A) Before starting out some time ago, I did some experiments with bup. I had a good experience with bup and high expectations for it. However, I found that quite a lot of performance was left on the…
<3
We've just added an applicability section, which explains a bit more what we do. We don't have any ELF specific heuristics [0]. https://github.com/elfshaker/elfshaker#applicability In summary, for manyclangs, we compile…
Author here. Compressed data is unlikely to work well in general, unless it never changes.
Author here. No architecture specific processing currently. Most of the magic happens in zstandard (hat tip to this amazing project). Please see our new applicability section which explains the result in a bit more…
Done, hopefully this is clearer. Please let us know if you see a way to improve it further: https://github.com/elfshaker/elfshaker/pull/60
(Disclosure: I work for Arm, opinions are my own) Author here. elfshaker itself does not have a dependency on any architecture to our knowledge. We support the architectures we have use of. Contributions to add missing…
Author here. The executables shipped in manyclangs are release builds! The catch is that manyclangs stores object files pre-link. Executables are materialized by relinking after they are extracted with elfshaker. The…
Author here. I've used bup, and elfshaker was partially inspired by it! It's great. However, during initial experiments on this project I found bup to be slow, taking quite a long time to snapshot and extract. I think…
Author here, I agree with xdfgh1112, please take care before using brand new software to store your backups!
Author here, this software is young, please don't use it for backups! But also, in general, it might not work well for your use case, and our use case is niche. Please give it a try before making assumptions about any…
Performing poorly with non-textual data happens for a a number of reasons. Binary data, when changed, often have a lot of 'non-local' changes in them. For example, a PSD file might well have a compression algorithm…
Author here, I don't think it would apply well to that scenario. elfshaker is good for manyclangs where we ship 2,000 revisions in one file (pack), so the cost of individual revision is amortized. If one build of…
Author here, it works particularly well for our presented use case because it has these properties: * There are many files, * Most of them don't change very often, * When they do change, the deltas of the binaries are…
Author here, I don't currently know how this compares to git-lfs. It it is possible git-lfs would perform quite well on the same inputs as elfshaker works on. If git-lfs does already work well for your use case I'd…
Author here. I'm unsure whether this would apply very well to cargo or not. If it has lots of pre-link object files, then maybe.
Author here. Maybe, it's a fun idea. I have toyed with providing a fuse filesystem for access to a pack but my time for completing this is limited at the moment.
Author here. We'd love this to be a thing, but this is young software, so we don't recommend relying on this as a single way of doing a backup for now. Bear in mind that our main use case is for things that you can…