Cyan4973
No user record in our sample, but Cyan4973 has activity below (stories or comments). Likely we have partial data — the full bulk-load will fill profiles in.
No user record in our sample, but Cyan4973 has activity below (stories or comments). Likely we have partial data — the full bulk-load will fill profiles in.
> Please be aware that this will significantly impact compression ratio. Is this a continuation from previous message ? Because multithreading mode of zstd doesn't impact its compression ratio. It's actually an…
I'm afraid I don't follow. Is there any code source that could be read ? The way to force the hash algorithm to actually wait for input is to use the previous hash to determine the start position of next hash's input.…
Following your suggestion, I went ahead and modified the scrambling formula. The crux of the problem is that the secret is allowed to be absolutely anything (it's user provided). As a consequence, it's a bad idea to…
Actually, making the next hash dependent on previous hash through the `seed` doesn't work for benchmarking latency. Several hashes only use the seed at the very end of calculation. Among them, CityHash, FarmHash,…
Thanks for suggestions @dragontamer. We are genuine when saying the algorithm is opened to suggestions, and can still change to improve its properties. Let's review yours : > my goal if I were to go through your code is…
> Why not optimize the function, and aim for only 256-bits of internal state (with 256-bits of entropy) ?? It's a lot more difficult to ensure that accumulators contain full-width entropy at all times. The trade-off in…
Well, if you believe a better scrambling operation is possible, you are certainly welcomed to suggest one. Considering feedbacks on the algorithm is one of the objectives of the test phase. If the issue is about the…
It's not exactly "losing information". We are not trying to regenerate original data, just make sure that all source bits can fairly influence the result. It's more a question of bit contribution. In the resulting…
According to the UMAC paper, the 32x32=>64 multiplication only contains 32-bit of entropy, even though it uses 64-bit space. That's understandable : most of the entropy will be in the middle of the register. That's…
wyhash is a close relative to mumhashv2, and feature approximately the same performance strength and weaknesses.
On Windows, I'm using Chocolatey package manager : https://chocolatey.org/ zstd is among the available packages.
zstd availability is becoming more and more common nowadays. See for example this tracker : https://repology.org/metapackage/zstd/versions
There is a zlib wrapper included in the project : https://github.com/facebook/zstd/tree/master/zlibWrapper
There is now a pzstd implementation, if you wish to compare it to pigz
The decompressor can receive a parameter to refuse compressed frames requiring more than a selected amount of memory.
Zstd pass faster over incompressible data. Expect something > 1 GB/s
zstd goes at > 1 GB/s on uncompressible data. It has some fast heuristics for such cases too.
MetroHash source code is impressively close to xxHash one, to the point that it seems to be a tweaked version of it. Yet, the author never mentions a single word about it in its source.
In all tests I know, xxh64 should be about 2x faster than xxh32 on x86_64 systems. Surprisingly, it doesn't match with the readme table.