Interesting ideas, but when it gets into utopian promises about being able to find all versions of a file ever on the web, it reminds me of the talk around RDF[1] and FOAF[2] in the late 90s/early 00s. Not to mention pre-Web hypertext theory, PGP, and any number of other things. All these things can be very useful patterns in tightly restricted, highly curated environments with specific use cases (whether or not they are part of the larger web). But they aren't panaceas and they don't address the underlying sources of the problems they are trying to work around, namely that all this unorganized junk on the web they aim to take control of is created by people and companies who are one or more of: lazy, sloppy, confused, conflicted, error-prone, avaricious, malicious, obnoxious, cruel, naive, desperate, or ignorant.
So, if we programmers are so advanced, why are we still using names to identify various programming language entities? Why aren't we giving the equivalent of UUIDs to Classes and Functions and other programming language entities? As it is now, there are version control systems that could well be confused by name collisions.
Because contrary to popular belief, source code is for humans. If we were after strict utilitarianism, we'd have switched to some MMIX-like abstract assembly with GUIDs as signifiers for all constructs.
Yeah, the idea of "let's serialize the ASTs and add names only when humans look at it" comes up all the time. There are also a few implementations, but editing it is just so different from text editing that the hurdle is quite big (and emulating text editing on top of it is really hard to get right, without loosing most of the benefits in the process).
Because "real words" kind of add an extra level of redundancy for the meaning.
Like Apple pie on a menu is way more convenient than a random number and then the description of the ingredient.
I don't think it aims at changing all URI.
It seems to be a way to obfuscate the address space in order to prevent attack by "educated guess". like if your UID is a monotonical growing number you can access another account by checking your predecessor.
httpS://mi6.gov.uk/007/my_profile
Don't laugh I actually have been ordered to build quite a few website with flaws like this against my advice where you could access others' profile's information to keep my job.
It seems to me a wrong technical answer to an education problem.
Using whatever kind of hash you want to give the attacker a 1/2^(whatever_big_power) protecting yourself from brute force attack to access a hidden (non authed) resource does not beat making sure people access information with the right credentials.
It clearly is a "statistical way" to remove the need for authentication if the "hash" is given by another channel.
Because statefull connections (https + auth + cookie + redirect) are expensive. (REST with auth is basically statefull).
It is a statistical bet relaying on the premise that your random number generator is not predictable, hence ... not testable :) Which is crazy.
Application: you are given a token (aka a random number or a hash) by a server and you use this normalized token to reference access to resources (either in input or output). It is not SciFi, it is in the rationale of the abstract.
Hash function outputs can be used [..] to a specific resource or to make URIs hard to guess for security reasons.
And with a hash big enough, you don't have to care about the risk of your session to be used... or maybe you should care.
It looks like something an Oauth2.0 fan boy could have come with.
Because that defeats the entire purpose of naming things. Sure, the way we name stuff isn't perfect. But it's way better than the alternative of giving it random names. They'll be unmemorable, ungoogleable, difficult to map mentally to concepts, etc.
This is actually one of the ideas underlying Microsoft's COM technology. Interfaces and classes had both names and guid identifiers. The guids were mostly hidden from programmers, and even when present they were #define'd to something more meaningful.
Lots of comments are asking what problem this solves.
First, content addressing is the best general solution to exactly-once message delivery in a distributed system. Examples of distributed message protocols include (ahem) email, HTTP and RSS. The alternative, UUIDs, is strictly worse in almost every way (which is to say, if you're currently using UUIDs for anything, consider switching to hashes of the content).
Second, content addressing guarantees the integrity of messages, meaning as long as you have the hash, you can get the data from anywhere. This is highly useful for mirroring and failover. Basically, we might be able to solve linkrot to a large degree, and make easier to mirror sites (especially locally, which is useful for high latency and offline).
I wrote up a short comparison of this protocol, along with a few alternatives a while back.[1] (Note that the IPFS comparison is incorrect, it should have an /ipfs/ path prefix.)
You just contact the central authorisation server and get a universal authorisation request authorisation.
Then, you take the universal authorisation request authorisation token and hand it off to the counter incrementing authorisation server which gives you a counter incrementing authorisation token.
Then, you take the counter incrementing authorisation token to your local counter incrementing server and ask it to increment the counter by 1.
In turn, your local counter incrementing server checks that your counter incrementing authorisation token was indeed signed by the central authorisation server and issues a counter incrementing operation to one of the 8 central counter incrementing servers.
Of course, your local counter incrementing server is itself authorised by the central authorisation server and the counter incrementing authorisation server to issue authoritative counter increment operations; so if its own local counter incrementing authorisation token has expired (5 minutes), it also has to do steps 1-3 to renew its authorisation to perform authoritative counter incrementing operations. This doesn't concern you, as all those requests are buffered so this section is purely informative.
Your own authorisation tokens also have expirations (5 minutes for the unviersal authorisation request authorisation token and ~10 seconds for the specific authorisation tokens) so you best issue your requests promptly and make sure to renew the tokens. Should your token expire before you have completed your counter increment request to your local counter increment server, perform steps 2-4 (or 1-4 if more than 5 minutes have elapsed since you acquired your universal authorisation request authorisation token).
Expect to receive a counter increment confirmation message within the next 2 full moons by carrier pigeon.
> You just contact the central authorisation server
No! No!! No!!!
First, we're doing distributed computing. There is no central authorization server. There are N authorization servers, and by the time you know for 100% sure whether the majority agree on whether you're authorized, someone has revoked the authorization. So take your authorization token, we'll write it down when you use it, and if you're de-authorized we'll adjust the counter after the fact to compensate, during our periodic audit.
But more importantly, the real problem is that your counter increment message was sent to a machine that has been taken offline by an errant backhoe and we don't know whether it took. Fortunately you sent the counter-increment message to three different servers, so once we have time to go through all the counter increment messages to de-duplicate we'll have your count, and the problem today is making sure that we have a full three copies of the message stored instead of just two.
Content addressing most definitely does NOT guarantee the integrity of a message, because hashes are non isomorphic homomorphisms. Cf. an article on "cryptographic hash"ing, for instance, which are hashes with strong statements about how hard it is to find distinct messages with the same hash, etc. .
I'm not sure I understand your criticism, but I should've clarified "content addressing using long, cryptographically secure hashes". Most of the major projects are using SHA-256.
The point is that hash functions of fixed bit length b have 2^b possible values, but are used to "represent" bit sequences of length longer than b, of which there are more than 2^b values. Collisions necessarily occur, but finding colliding values isn't necessarily easy (cf. cryptographic hashes), which means if you want to tamper with a message, which has an associated hash, without being detected, you're constrained by the hash function to tamperings which produce the same hash.
I had to look it up, and found info_hash is the SHA-1 of the 'info' section of the metainfo file, which lists the files and the SHA-1 hashes of each of their pieces. So you are asking how many torrents have exactly the same file data but different metadata. So yes, interesting question.
If two torrents differ only in metadata, will any bittorrent clients actually allow the data be shared (without manual intervention like symlinking files)?
Which basically means that it guarantees the integrity of message, where "guarantee" is defined as computational infeasibility to produce different messages with the same hash value. You know this stuff, why are you arguing semantics?
I hear you regarding infeasibility, though right now there's another article about content hash collisions being used to distribute a tampered-with Mint Linux image (MD5 collisions).
It's "easier" to get the computational resources to fabricate a collision there, of course, but who is to say how much compute power state level adversaries bring to bear?
If you use a message hash as your unique identifier for exactly-once delivery, your messaging system won't let a user send the same message twice, even if they mean to.
You could wrap the user's message in some metadata including a uuid to ensure the overall content is distinct from anytime the user previously posted that message. But now you're back at uuids, which you say you don't like.
Counters have a host of other problems, like making sure that you never issue the same counter twice in a system without a central authority (or just one that doesn't desire a single point of failure)
You need to think about what makes messages distinct. Content addressing will help you as long as you have a rigorous definition of identity, which you might otherwise be doing ad-hoc when you assign UUIDs.
If the same message is meaningful at different points in time, your hashed content should include a timestamp (and of course you need to choose an appropriate level of precision). If your message is meaningful in different contents, it needs to reference that context somehow (ideally by hash).
The payoff for doing this well is 1. an elimination of double-sent emails, double-posts, etc. (when the software might not know if it went through the first time), and 2. the ability to deduplicate across a network partition, for example if a user makes the same change on their computer and on their phone and then syncs them.
... except wallclock timestamps for ordering of events in a distributed system is a terrible idea in most cases for the simple reason that (a) it's not a (even close to) guaranteed unique event identifier (the same time happens in many locations), (b) you usually don't have a clock of sufficient precision to even reliably establish a total order of distinct moments and (c) even if you have, that order does not necessarily preserve causality.
Now, UUIDs don't solve (b) or (c) either, but at least they actually give you a reliable identifier for events.
They're events, but no ordering is assumed, guaranteed, promised, implied, or enforced. The timestamps, if you choose to use them, are only for determining message identity (equality) and thus can be as accurate or inaccurate as relevant for your application.
You don't have to use timestamps if they don't make sense. For example, what is the "identity" of a message in an instant messaging program? If you just use the message content, then duplicate messages don't work. It would make sense to include the sender and the time written (which is different from the time sent!) as well. You might also include the recipient, depending.
Note that instead of using timestamps, you could use the hash of the message being replied to. However, IM software usually doesn't have a system for real replies, so doing that properly probably isn't worthwhile. (Edit: however, hashes are very useful for systems like email, forums, etc., where reply targets are explicitly defined.)
If you do want ordering, BYO. Content addressing and UUIDs are equivalent in that regard.
> The timestamps, if you choose to use them, are only for determining message identity (equality)
Which simply doesn't work because timestamps are not unique to location. Every clock experiences every timestamp, and thus potentially assigns any timestamp to an event, so multiple clocks can end up assigning the same timestamp to distinct events, and thus identifying them.
> Note that instead of using timestamps, you could use the hash of the message being replied to.
If your message isn't unique, the message you are replying to probably isn't either, so it's useless to use it as an identifier.
> If you do want ordering, BYO. Content addressing and UUIDs are equivalent in that regard.
No, they are not, at all, they aren't even the same kind of thing. UUIDs are unique identifiers, you generate them to label things that couldn't be distinguished without them. Content addressing is nothing more than summarizing a given object in a way that (almost certainly) preserves identity. So, one creates distinction, the other maintains identity.
> Which simply doesn't work because timestamps are not unique to location.
Yes, I know. This is okay. Whatever time you think you wrote the message at is fine. Even if your clock is off. Even if you lie. That is (part of) the message content (in this case).
The time the message is sent is meaningless and can't be used, I agree. The time the message was written, or created, or "thought to be created" is what matters (or doesn't).
> If your message isn't unique, the message you are replying to probably isn't either, so it's useless to use it as an identifier.
Git does this and works great. If your entire message history (via hash chain) is identical, then you really do have duplicate messages.
> UUIDs are unique identifiers, you generate them to label things that couldn't be distinguished without them. Content addressing is nothing more than summarizing a given object in a way that (almost certainly) preserves identity. So, one creates distinction, the other maintains identity.
I agree, you're right. However, this is a feature.
> Yes, I know. This is okay. Whatever time you think you wrote the message at is fine. Even if your clock is off. Even if you lie. That is (part of) the message content (in this case).
Yes, it's part of its content, sure. That doesn't make it an identity of the event that originated it.
> The time the message is sent is meaningless and can't be used, I agree. The time the message was written, or created, or "thought to be created" is what matters (or doesn't).
But still doesn't provide an identity of the event.
> Git does this and works great. If your entire message history (via hash chain) is identical, then you really do have duplicate messages.
But that is simply because you have defined message identity in this context to mean just that, which is fine because of the semantics of the "messages" that git is made to manage. What it does not is provide identity of commit events. You can not, for example, implement a distributed counter in git that works by committing changes to a file that add a line with a number in it. If two git instances commit adding the same number based on the same parents at the same time with the same authorship information [...], the merge result might not reflect all counter increment operations that happened.
> I agree, you're right. However, this is a feature.
Well, it just is what it is, and each has its uses, but they are mostly orthogonal. It's just that when a UUID is part of an identity, summarizing that identity is easier done by dropping everything but the UUID instead of hashing everything.
> You can not, for example, implement a distributed counter in git that works by committing changes to a file that add a line with a number in it. If two git instances commit adding the same number based on the same parents at the same time with the same authorship information [...], the merge result might not reflect all counter increment operations that happened.
Yes. That's why content has to be defined at the application level, based on what the application is trying to accomplish. If absolutely every message is logically unique, then under a content addressing system each message needs to include some random data, which will result in random hashes equivalent to UUIDs. That is the worst case scenario for content addressing.
Even so, it's still "as good or better" (aside from performance concerns, which I don't mean to dismiss).
Content hashing doesn't rely on timestamps to act as a unique identifier. It relies on all of the content, which may (or may not) include a timestamp, depending on your application. To the degree that collisions occur, that's a feature, not a bug. If you get undesirable collisions, you aren't hashing the right data.
Edit: to clarify, content hashes are just like UUIDs except you can get useful collisions if you want them.
The context here is events originated by a single actor, so wallclock is much more reasonable than in the general distributed system case. You might still be worried about the user accessing your system simultaneously from multiple devices, so a randomly generated ID is probably still useful.
First, it's usually impossible to tell actors apart, so it's not particularly useful to build an algorithm that depends on doing so.
Second, while the actor might be the same, the time source usually is not, so it still doesn't help you.
Third, if actor and time source are indeed the same, you really are using a counter, the fact that it's correlated with wallclock time is coincidental.
Piling on, even if the actor and time source are the same, you have to be really careful about how your use "wall clock time", because if you're just asking the OS what time it is, a lot of platforms allow the clock to move backwards, not move, etc.
Ok, so the simple example you gave is very abstract. Alice and bob may have a great time talking on the phone but I felt no closer to understanding its purpose.
I do understand the purpose in requesting a resource from Alice that has no ip address associated to it. Say Bob and Alice are part of a network. Alice saves s file on one of the computers but has no idea which one. Bob wants that file. Hash Identifier protocol provides a standardized way to retrieve it. Using it, Bob makes a request to sll conputers and only one responds.
The RFC felt far more boring than this topic is. Technologies such as magnet uris and IPFS are your competition. I would argue that security is not the purpose for this though it can be used in conjuction. I believe this is far more effective for distributed systems
It seems that the authors have decent ideas, but don't have much experience with the subject matter. If you look at the assigned identifiers in the binary scheme, the authors believe that 32-bit truncated cryptographic hash functions are useful. 64-bit truncated cryptographic hash functions are essentially useless from a security standpoint, so the 32- and 64-bit variants would be better served by CRC-32C and CRC-64ECMA. If your standard contains a 64-bit truncated cryptographic hash, people without subject matter expertise will assume that it provides more than a modicum of security.
Also, the authors apparently vastly underestimate the utility of using Merkle trees. I used to be an engineer for LimeWire. Gnutella used SHA-1 as identifiers for the content. Unfortunately, this means only being able to integrity-check an entire large file instead of small pieces, or having to get a Merkle tree root out-of-band. LimeWire just got the Merkle tree root (plus one row of the tree) from the first peer it contacted for download data. This represents a simple denial-of-service poisoning attack where the attacker hands out Merkle trees corresponding to something like 10% of the blocks being corrupted. The clients would then repeatedly request the "corrupted" blocks from peers, until giving up, notifying the user that the download was corrupted, and most likely keeping around a file that had sections that weren't integrity checked. (Corruption really does happen. The TCP checksum is rather weak.) If they had used the Merkle tree root as the file identifier instead, then there would be no opportunity to trick the client into incorrectly associating the wrong Merkle tree with a given file.
If you're going to define a hash URI scheme, please incorporate Sakura trees (a provably secure hash tree scheme) of degree 2 with a fixed leaf block size. Leaving the block size variable leads to the Bittorrent problem where a single set of identical files has multiple identifiers and clients using hashing at one granularity can't share information with clients using hashing at another granularity. Merkle trees with a single standard leaf block size allow different levels of the tree to be shared to in effect give different granularities, without dividing resources due to the multiple identifier problem.
Also, in the case where there are 2^N + 1 blocks (or any other case where you'd be tempted to "optimize" by skipping a node at that level), please have a re-hashing node at each level for all blocks. This means that the final block (along with at most one extra hash per tree level) constitutes a cryptographic proof of the file length corresponding to the tree root. Otherwise, in order to avoid certain denial of service attacks, you also need to always put the length of the file as part of the URI, or the first client needs to send the entire bottom row of the Sakura tree.
Note that the Bittorrent Merkle tree format is broken in the same way that the original Gnutella tiger tree proposal was broken (fixed before implementation in the Gnutella case). Use Sakura trees. They're provably as secure as the hash function you use.
If you look at the assigned identifiers in the binary scheme, the authors believe that 32-bit truncated cryptographic hash functions are useful.
No they don't. From Section 2:
The sha-256 algorithm as specified in [SHA-256] is mandatory to
implement; that is, implementations MUST be able to generate/send and
to accept/process names based on a sha-256 hash. However,
implementations MAY support additional hash algorithms and MAY use
those for specific names, for example, in a constrained environment
where sha-256 is non-optimal or where truncated names are needed to
fit into corresponding protocols (when a higher collision probability
can be tolerated).
Truncated hashes MAY be supported. When a hash value is truncated,
the name MUST indicate this. Therefore, we use different hash
algorithm strings in these cases, such as sha-256-32 for a 32-bit
truncation of a sha-256 output. A 32-bit truncated hash is
essentially useless for security in almost all cases but might be
useful for naming. With current best practices [RFC3766], very few,
if any, applications making use of names with less than 100-bit
hashes will have useful security properties.
See also: https://lkml.org/lkml/2010/10/28/287 where Linus Torvalds says 12 hex digits (96 bits) is pretty much the minimum short-hash for the Linux kernel commit history.
Extremely short hashes can be useful briefly for manually transcribing between devices, as long as you immediately "resolve" them back into a longer form, before new collisions can happen. But this is more on par with clicking "I'm feeling lucky" than creating a link. :)
I came here to talk about this. I deal with data management for a bunch of physics and astronomy experiments it's quite typical to have many millions of files for any given experiment (say, 1PB of storage for a medium size experiment, 10 million 100MB files). CERN's experiments would easily have billions of files, so I was thinking 64 bits would be a minimal truncation, but 96 is probably more reasonable.
80 comments
[ 2.4 ms ] story [ 144 ms ] threadIt's not the exact same specification, but I imagine these two concepts are similar?
[0]: https://joearms.github.io/2015/03/12/The_web_of_names.html
[1] https://en.wikipedia.org/wiki/Resource_Description_Framework [2] https://en.wikipedia.org/wiki/FOAF_(ontology)
One of the magical things about the web is that you have the freedom to distribute information as you wish, and DNS namespace scopes everything.
The part where he talks about "the entropy reverser" is ~32 minutes in [1], related to the article linked by parent.
[1] "The Mess We're In" by Joe Armstrong (September, 2014) https://youtu.be/lKXe3HUG2l4?t=32m5s
I got afraid.
example: http://www.lamdu.org/
Like Apple pie on a menu is way more convenient than a random number and then the description of the ingredient.
I don't think it aims at changing all URI.
It seems to be a way to obfuscate the address space in order to prevent attack by "educated guess". like if your UID is a monotonical growing number you can access another account by checking your predecessor.
httpS://mi6.gov.uk/007/my_profile
Don't laugh I actually have been ordered to build quite a few website with flaws like this against my advice where you could access others' profile's information to keep my job.
It seems to me a wrong technical answer to an education problem.
Using whatever kind of hash you want to give the attacker a 1/2^(whatever_big_power) protecting yourself from brute force attack to access a hidden (non authed) resource does not beat making sure people access information with the right credentials.
It clearly is a "statistical way" to remove the need for authentication if the "hash" is given by another channel.
Because statefull connections (https + auth + cookie + redirect) are expensive. (REST with auth is basically statefull).
It is a statistical bet relaying on the premise that your random number generator is not predictable, hence ... not testable :) Which is crazy.
Application: you are given a token (aka a random number or a hash) by a server and you use this normalized token to reference access to resources (either in input or output). It is not SciFi, it is in the rationale of the abstract.
Hash function outputs can be used [..] to a specific resource or to make URIs hard to guess for security reasons.
And with a hash big enough, you don't have to care about the risk of your session to be used... or maybe you should care.
It looks like something an Oauth2.0 fan boy could have come with.
First, content addressing is the best general solution to exactly-once message delivery in a distributed system. Examples of distributed message protocols include (ahem) email, HTTP and RSS. The alternative, UUIDs, is strictly worse in almost every way (which is to say, if you're currently using UUIDs for anything, consider switching to hashes of the content).
Second, content addressing guarantees the integrity of messages, meaning as long as you have the hash, you can get the data from anywhere. This is highly useful for mirroring and failover. Basically, we might be able to solve linkrot to a large degree, and make easier to mirror sites (especially locally, which is useful for high latency and offline).
I wrote up a short comparison of this protocol, along with a few alternatives a while back.[1] (Note that the IPFS comparison is incorrect, it should have an /ipfs/ path prefix.)
[1] https://bentrask.com/?q=hash://sha256/1e9a6b770ef9f1ca894af4...
Then, you take the universal authorisation request authorisation token and hand it off to the counter incrementing authorisation server which gives you a counter incrementing authorisation token.
Then, you take the counter incrementing authorisation token to your local counter incrementing server and ask it to increment the counter by 1.
In turn, your local counter incrementing server checks that your counter incrementing authorisation token was indeed signed by the central authorisation server and issues a counter incrementing operation to one of the 8 central counter incrementing servers.
Of course, your local counter incrementing server is itself authorised by the central authorisation server and the counter incrementing authorisation server to issue authoritative counter increment operations; so if its own local counter incrementing authorisation token has expired (5 minutes), it also has to do steps 1-3 to renew its authorisation to perform authoritative counter incrementing operations. This doesn't concern you, as all those requests are buffered so this section is purely informative.
Your own authorisation tokens also have expirations (5 minutes for the unviersal authorisation request authorisation token and ~10 seconds for the specific authorisation tokens) so you best issue your requests promptly and make sure to renew the tokens. Should your token expire before you have completed your counter increment request to your local counter increment server, perform steps 2-4 (or 1-4 if more than 5 minutes have elapsed since you acquired your universal authorisation request authorisation token).
Expect to receive a counter increment confirmation message within the next 2 full moons by carrier pigeon.
Super simple stuff.
No! No!! No!!!
First, we're doing distributed computing. There is no central authorization server. There are N authorization servers, and by the time you know for 100% sure whether the majority agree on whether you're authorized, someone has revoked the authorization. So take your authorization token, we'll write it down when you use it, and if you're de-authorized we'll adjust the counter after the fact to compensate, during our periodic audit.
But more importantly, the real problem is that your counter increment message was sent to a machine that has been taken offline by an errant backhoe and we don't know whether it took. Fortunately you sent the counter-increment message to three different servers, so once we have time to go through all the counter increment messages to de-duplicate we'll have your count, and the problem today is making sure that we have a full three copies of the message stored instead of just two.
Wonder, if anyone had already did a research how many different torrents with the same `info_hash` one can find out there.
If two torrents differ only in metadata, will any bittorrent clients actually allow the data be shared (without manual intervention like symlinking files)?
You guys are worrying about the wrong thing. Any hash considered secure will have zero collisions, at least in your light cone for trillions of years.
If current hashes are shown to have weaknesses they will be deprecated, probably before even a single collision takes place.
SHA-1 is "cryptographically broken": it's estimated that you can produce collision with about 2^60 operations, however nobody has produced them yet.
It's "easier" to get the computational resources to fabricate a collision there, of course, but who is to say how much compute power state level adversaries bring to bear?
who is to say how much compute power state level adversaries bring to bear
Not enough to break modern hash functions.
You could wrap the user's message in some metadata including a uuid to ensure the overall content is distinct from anytime the user previously posted that message. But now you're back at uuids, which you say you don't like.
If the same message is meaningful at different points in time, your hashed content should include a timestamp (and of course you need to choose an appropriate level of precision). If your message is meaningful in different contents, it needs to reference that context somehow (ideally by hash).
The payoff for doing this well is 1. an elimination of double-sent emails, double-posts, etc. (when the software might not know if it went through the first time), and 2. the ability to deduplicate across a network partition, for example if a user makes the same change on their computer and on their phone and then syncs them.
Now, UUIDs don't solve (b) or (c) either, but at least they actually give you a reliable identifier for events.
You don't have to use timestamps if they don't make sense. For example, what is the "identity" of a message in an instant messaging program? If you just use the message content, then duplicate messages don't work. It would make sense to include the sender and the time written (which is different from the time sent!) as well. You might also include the recipient, depending.
Note that instead of using timestamps, you could use the hash of the message being replied to. However, IM software usually doesn't have a system for real replies, so doing that properly probably isn't worthwhile. (Edit: however, hashes are very useful for systems like email, forums, etc., where reply targets are explicitly defined.)
If you do want ordering, BYO. Content addressing and UUIDs are equivalent in that regard.
Which simply doesn't work because timestamps are not unique to location. Every clock experiences every timestamp, and thus potentially assigns any timestamp to an event, so multiple clocks can end up assigning the same timestamp to distinct events, and thus identifying them.
> Note that instead of using timestamps, you could use the hash of the message being replied to.
If your message isn't unique, the message you are replying to probably isn't either, so it's useless to use it as an identifier.
> If you do want ordering, BYO. Content addressing and UUIDs are equivalent in that regard.
No, they are not, at all, they aren't even the same kind of thing. UUIDs are unique identifiers, you generate them to label things that couldn't be distinguished without them. Content addressing is nothing more than summarizing a given object in a way that (almost certainly) preserves identity. So, one creates distinction, the other maintains identity.
Yes, I know. This is okay. Whatever time you think you wrote the message at is fine. Even if your clock is off. Even if you lie. That is (part of) the message content (in this case).
The time the message is sent is meaningless and can't be used, I agree. The time the message was written, or created, or "thought to be created" is what matters (or doesn't).
> If your message isn't unique, the message you are replying to probably isn't either, so it's useless to use it as an identifier.
Git does this and works great. If your entire message history (via hash chain) is identical, then you really do have duplicate messages.
> UUIDs are unique identifiers, you generate them to label things that couldn't be distinguished without them. Content addressing is nothing more than summarizing a given object in a way that (almost certainly) preserves identity. So, one creates distinction, the other maintains identity.
I agree, you're right. However, this is a feature.
Yes, it's part of its content, sure. That doesn't make it an identity of the event that originated it.
> The time the message is sent is meaningless and can't be used, I agree. The time the message was written, or created, or "thought to be created" is what matters (or doesn't).
But still doesn't provide an identity of the event.
> Git does this and works great. If your entire message history (via hash chain) is identical, then you really do have duplicate messages.
But that is simply because you have defined message identity in this context to mean just that, which is fine because of the semantics of the "messages" that git is made to manage. What it does not is provide identity of commit events. You can not, for example, implement a distributed counter in git that works by committing changes to a file that add a line with a number in it. If two git instances commit adding the same number based on the same parents at the same time with the same authorship information [...], the merge result might not reflect all counter increment operations that happened.
> I agree, you're right. However, this is a feature.
Well, it just is what it is, and each has its uses, but they are mostly orthogonal. It's just that when a UUID is part of an identity, summarizing that identity is easier done by dropping everything but the UUID instead of hashing everything.
Yes. That's why content has to be defined at the application level, based on what the application is trying to accomplish. If absolutely every message is logically unique, then under a content addressing system each message needs to include some random data, which will result in random hashes equivalent to UUIDs. That is the worst case scenario for content addressing.
Even so, it's still "as good or better" (aside from performance concerns, which I don't mean to dismiss).
Edit: to clarify, content hashes are just like UUIDs except you can get useful collisions if you want them.
Second, while the actor might be the same, the time source usually is not, so it still doesn't help you.
Third, if actor and time source are indeed the same, you really are using a counter, the fact that it's correlated with wallclock time is coincidental.
I do understand the purpose in requesting a resource from Alice that has no ip address associated to it. Say Bob and Alice are part of a network. Alice saves s file on one of the computers but has no idea which one. Bob wants that file. Hash Identifier protocol provides a standardized way to retrieve it. Using it, Bob makes a request to sll conputers and only one responds.
The RFC felt far more boring than this topic is. Technologies such as magnet uris and IPFS are your competition. I would argue that security is not the purpose for this though it can be used in conjuction. I believe this is far more effective for distributed systems
Also, the authors apparently vastly underestimate the utility of using Merkle trees. I used to be an engineer for LimeWire. Gnutella used SHA-1 as identifiers for the content. Unfortunately, this means only being able to integrity-check an entire large file instead of small pieces, or having to get a Merkle tree root out-of-band. LimeWire just got the Merkle tree root (plus one row of the tree) from the first peer it contacted for download data. This represents a simple denial-of-service poisoning attack where the attacker hands out Merkle trees corresponding to something like 10% of the blocks being corrupted. The clients would then repeatedly request the "corrupted" blocks from peers, until giving up, notifying the user that the download was corrupted, and most likely keeping around a file that had sections that weren't integrity checked. (Corruption really does happen. The TCP checksum is rather weak.) If they had used the Merkle tree root as the file identifier instead, then there would be no opportunity to trick the client into incorrectly associating the wrong Merkle tree with a given file.
If you're going to define a hash URI scheme, please incorporate Sakura trees (a provably secure hash tree scheme) of degree 2 with a fixed leaf block size. Leaving the block size variable leads to the Bittorrent problem where a single set of identical files has multiple identifiers and clients using hashing at one granularity can't share information with clients using hashing at another granularity. Merkle trees with a single standard leaf block size allow different levels of the tree to be shared to in effect give different granularities, without dividing resources due to the multiple identifier problem.
Also, in the case where there are 2^N + 1 blocks (or any other case where you'd be tempted to "optimize" by skipping a node at that level), please have a re-hashing node at each level for all blocks. This means that the final block (along with at most one extra hash per tree level) constitutes a cryptographic proof of the file length corresponding to the tree root. Otherwise, in order to avoid certain denial of service attacks, you also need to always put the length of the file as part of the URI, or the first client needs to send the entire bottom row of the Sakura tree.
Note that the Bittorrent Merkle tree format is broken in the same way that the original Gnutella tiger tree proposal was broken (fixed before implementation in the Gnutella case). Use Sakura trees. They're provably as secure as the hash function you use.
No they don't. From Section 2:
See also: https://lkml.org/lkml/2010/10/28/287 where Linus Torvalds says 12 hex digits (96 bits) is pretty much the minimum short-hash for the Linux kernel commit history.
Extremely short hashes can be useful briefly for manually transcribing between devices, as long as you immediately "resolve" them back into a longer form, before new collisions can happen. But this is more on par with clicking "I'm feeling lucky" than creating a link. :)