A UUID generated or used by any of Microsoft's APIs or tools named with "Guid" are 100% standard UUIDs. It is not possible to create a a UUID with these APIs or tools that does not conform to the standard.
GUIDs are a particular implementation of UUID, yes. The big difference is that GUID tends to be upper-case and little-endian (except for the big-endian bit), whereas UUIDs tend to be lower-case and big-endian.
UUIDs and GUIDs are far too complicated, personally I don't like using them. There are multiple "versions" (really, generation algorithms) of UUID and GUID, each with their own problems:
* Some types of UUID uniquely identify the machine they were generated on (one version contains the MAC address + current time, another contains the POSIX UID/GID + domain name) - this got Microsoft into hot water in the 1990s when Word added GUIDs to documents, which meant you could trace documents Stasi-style
* Some types of UUID are based on insecure hashing algorithms (MD5 and SHA1)
* Some types of UUID are namespaced, because everything needs namespaces, obviously
* There's a specially reserved type for Microsoft to use for special COM objects
There's only one mode you should actually use, which is the random bits.
UUIDs and GUIDs also have a weird spacing of dashes. You'd expect three dashes, splitting it into a sequence of 4-byte chunks, but no: it's split into 4-2-2-2-6, for some reason. And which chunk it is matters, because different chunks have different endianness. Some of them are considered numbers, some of them bytes, even though they all look the same. Some of them have special significance (it contains two different version numbers!), with no especially obvious rhyme or reason to their placement. Oh, and the endianness is implementation-defined: GUIDs are partly "native" endian (usually little-endian, then), partly big-endian, whereas UUIDs are typically big-endian. How do you tell them apart? Well, GUIDs are usually written in capitals, and UUIDs are usually written in lowercase.
I just use 16 random bytes encoded in hexadecimal, separated by three dashes at 4-byte increments. No hashing algorithms, versions, endianness issues, namespacing, severe privacy problems, just random bytes. It's not only simpler, it has more bits of entropy, and is easier to generate.
I also treat a UUID as a string of bytes. Go ahead and render it any way you like, but there are no endianness issues. The bytes are random and meaningless.
Also be careful of what you think is 'random'. Calling the OS random-number-generator may not be random enough. I'd recommend in every case to use the system library to generate a UUID.
No, endianness need not mean anything, when you are using uuids for your own purposes. Go ahead and identify a resource by this id. When it is returned to you, it will be as you sent it. No matter what (pointless) swapping and unswapping was done by the other participant. They will return the string the same as they got it (or they have a bug).
To have endianness implies scalar subfields, which implies meaningful values in those fields. This goes back to when the fields had meaning (mac address, timestamp and so on). Nowadays they are random bytes and the breakdown into scalar parts has no useful purpose.
Now, you can print UUIDs any way you like. And if you print them different places with different rendering libraries they can look different, sure. Don't confuse that with the underlying value, which is a random string.
> No, endianness need not mean anything, when you are using uuids for your own purposes. Go ahead and identify a resource by this id. When it is returned to you, it will be as you sent it. No matter what (pointless) swapping and unswapping was done by the other participant. They will return the string the same as they got it (or they have a bug).
If you're passing around the string representation, then endianness isn't an issue, sure. But you need to care about it if you store UUIDs efficiently as bytes.
> To have endianness implies scalar subfields, which implies meaningful values in those fields. This goes back to when the fields had meaning (mac address, timestamp and so on). Nowadays they are random bytes and the breakdown into scalar parts has no useful purpose.
Even if you're only using random UUIDs, you need to worry about this, because UUIDs always contain a version and variant number.
> Now, you can print UUIDs any way you like. And if you print them different places with different rendering libraries they can look different, sure. Don't confuse that with the underlying value, which is a random string.
The string is only mostly random. It contains version information.
If you're passing around the string representation, then endianness isn't an issue, sure. But you need to care about it if you store UUIDs efficiently as bytes
Huh? Strings are stored and transmitted as bytes, there is no need to interpret the subfields as any given endianness if the whole thing is stored as 16 bytes.
The typical string/text representation of a UUID is 36 bytes (16 bytes x 2 (hex encoding) + 4 (dashes). Or more if you want to include the NUL terminator, enclosing {}s, a length prefix, or whatever else.
Alternatively you can interpret the hexadecimal encoding of the subfields to raw values, and compact things into a single 16 byte / 128-bit value, which I'm assuming is what's being referred to by "storing UUIDs efficiently as bytes".
To perform this compacting requires some interpretation - if only from hexadecimal to bytes. In implementing this, you might feed the entire first subfield to your hex parsing function. The result, however, will be an integer with more bits than a byte, at which point you need to care about endianness.
You can choose to instead parse two hex characters as a time and treat each as a byte. You could argue that this is "not" interpreting the subfield as any given endianness - but this is semantically identical to interpreting the field as big endian, so I'd argue that you are.
The great advantage of being standardized is that other people have thought about these issues and dealt with them in libraries. Yes there are many bad ways to generate them, but your library should offer the right way to generate them. Yes the string format is weird, but why would you write that by hand? Let the library print it, let the library parse it. Want to store it in a database? There's a column type for that. Want to send it over the network? There's probably a datatype for it in whatever protocol you're using (whereas with your approach you immediately have endianness issues).
I can't look-up the details right now, but we ran into some issues when we wanted to use system uuids (via dmidecode) to identify servers and pxeboot them. Apparently, HP server uuids are (subtly) different through dmidecode and via pxeboot... And it could be endianness actually - I did not check why it we different...
> The great advantage of being standardized is that other people have thought about these issues and dealt with them in libraries.
Issues that don't exist if you don't use a hideously complex format like UUID in the first place.
> Yes there are many bad ways to generate them, but your library should offer the right way to generate them.
Yes, a UUID library will have a way to generate the random kind of UUID, but it's telling that you need a library for it.
You can generate 16 random bytes without a library, by using fopen() and fread().
> Yes the string format is weird, but why would you write that by hand? Let the library print it, let the library parse it.
You shouldn't need a library to handle your identifiers. If you do, your identifiers are too complicated.
> Want to store it in a database? There's a column type for that. Want to send it over the network? There's probably a datatype for it in whatever protocol you're using (whereas with your approach you immediately have endianness issues).
With the approach of random bytes, there's at least a consistent endianness throughout. And you can solve the endianness issue by treating it as a byte string. Job done.
Yes, the UUID 5 (SHA1 + namespace) is fantastically useful because it is deterministic. They can be a function of your own internal IDs, rather than trying to synchronize clocks or random seeds across systems, and still won't conflict with UUIDs from other vendors using a similar system.
MD5 and SHA-1 being broken only matter if you let users choose your keys, to which the answer is don't do that.
I can deal with a machine crashing, I can not deal with an invalid database caused by GUIDs intersecting. Thus I actually want to avoid GUID intersections more than I care about a single machine's valid ram.
> GUID generation algorithm 4 fills the GUID with 122 random bits. The odds of two GUIDs colliding are therefore one in 2^122, which is a phenomenally small number.
Another thing to consider, is that due to the birthday paradox once you build 2.7 * 10^18 GUID, the probability that you have at lest a collision is bigger than 50%. And 2.7 * 10^18 is only 2^61.2.
Well, if they did understand, they would say the chance is ~1 in 2^61, not 1 in 2^122, and they would've based the math comparison to RAM failure on 2^61, which changes the comparison entirely.
In any case, I should have been more clear. I'm not necessarily worried about this case. But if you design systems and miss a factor of a square root, you tend to break things that we're all relying on to work.
If you generated 2.7 * 10^18 GUIDs (and obviously stored them all, otherwise the birthday paradox is not relevant), you also used up 43 exabytes (=1 million terrabytes) of storage. I wonder which problem you will encounter first...
Note that at 128 bits per GUID (not including any storage overhead), 2.7x10^18 GUIDs would take up 43.2 exabytes of storage. That's about 10 million 4TB drives.
It's going to be a while before the birthday paradox is really going to be problematic.
Duplicate GUID:s matter only if there is actual comparison.
There may be duplicate GUID:s used in different contexts and that's just fine. If there is single system and context where storing just GUID:s takes several exabytes we may start worrying.
Thanks for saying this. The birthday paradox calculation is far more relevant to determining collision risks. Rule of thumb: if you have N random IDs, then after sqrt(N) IDs are generated there's a 50% probability of a single collision. Fortunately the sqrt(2^122) is still 2^62, or a very large number of IDs. But 64 bit random IDs have a collision after only 2^32, or 4 billion, and that has happened in practice in several systems.
Be careful there. Parts of Europe including here (the UK) officially use short scale like the US.
When looking at historical figures it is important to be extra careful as scale use has flipped back and forth over time in places. "Historical" doesn't go as far back as you think either: short scale becoming the standard number naming convention in the UK happened in 1974 so there are still people alive who use long scale and remember it being the most common form.
To really confuse things some places use a half-way house of "short scale with milliard"...
It is safer to stick with "scientific" prefixes (kilo, mega, giga, tera, ...) as they are consistently interpreted the same way except where someone is being deliberately difficult (for "deliberately difficult" read "just plain wrong"). It sometimes sounds odd referring to things like "giga pounds" instead of "billions of pounds" (or "thousand millions of pounds" or "milliards of pounds") but it reduces the risk of misinterpretation and anyone who doesn't understand probably wouldn't truly understand any of the above terms without explanation.
46 comments
[ 3.0 ms ] story [ 89.2 ms ] threadhttps://en.wikipedia.org/wiki/Globally_unique_identifier#Bin...
http://stackoverflow.com/questions/246930/is-there-any-diffe...
A UUID generated or used by any of Microsoft's APIs or tools named with "Guid" are 100% standard UUIDs. It is not possible to create a a UUID with these APIs or tools that does not conform to the standard.
* Some types of UUID uniquely identify the machine they were generated on (one version contains the MAC address + current time, another contains the POSIX UID/GID + domain name) - this got Microsoft into hot water in the 1990s when Word added GUIDs to documents, which meant you could trace documents Stasi-style
* Some types of UUID are based on insecure hashing algorithms (MD5 and SHA1)
* Some types of UUID are namespaced, because everything needs namespaces, obviously
* There's a specially reserved type for Microsoft to use for special COM objects
There's only one mode you should actually use, which is the random bits.
UUIDs and GUIDs also have a weird spacing of dashes. You'd expect three dashes, splitting it into a sequence of 4-byte chunks, but no: it's split into 4-2-2-2-6, for some reason. And which chunk it is matters, because different chunks have different endianness. Some of them are considered numbers, some of them bytes, even though they all look the same. Some of them have special significance (it contains two different version numbers!), with no especially obvious rhyme or reason to their placement. Oh, and the endianness is implementation-defined: GUIDs are partly "native" endian (usually little-endian, then), partly big-endian, whereas UUIDs are typically big-endian. How do you tell them apart? Well, GUIDs are usually written in capitals, and UUIDs are usually written in lowercase.
I just use 16 random bytes encoded in hexadecimal, separated by three dashes at 4-byte increments. No hashing algorithms, versions, endianness issues, namespacing, severe privacy problems, just random bytes. It's not only simpler, it has more bits of entropy, and is easier to generate.
Also be careful of what you think is 'random'. Calling the OS random-number-generator may not be random enough. I'd recommend in every case to use the system library to generate a UUID.
Why not avoid UUID altogether? It is an overly complicated format. It is simpler to just generate 16 random bytes and represent them as you wish.
Beware that 16 random bytes, at least currently, is not a valid UUID. You have to include version and variant numbers.
> Go ahead and render it any way you like, but there are no endianness issues. The bytes are random and meaningless.
Endianness absolutely matters. If you don't encode it and decode it correctly from the binary formats, you will end up with a different identifier.
> Also be careful of what you think is 'random'. Calling the OS random-number-generator may not be random enough.
If it's just a regular PRNG, sure. What you want is a CSPRNG, like /dev/urandom.
To have endianness implies scalar subfields, which implies meaningful values in those fields. This goes back to when the fields had meaning (mac address, timestamp and so on). Nowadays they are random bytes and the breakdown into scalar parts has no useful purpose.
Now, you can print UUIDs any way you like. And if you print them different places with different rendering libraries they can look different, sure. Don't confuse that with the underlying value, which is a random string.
If you're passing around the string representation, then endianness isn't an issue, sure. But you need to care about it if you store UUIDs efficiently as bytes.
> To have endianness implies scalar subfields, which implies meaningful values in those fields. This goes back to when the fields had meaning (mac address, timestamp and so on). Nowadays they are random bytes and the breakdown into scalar parts has no useful purpose.
Even if you're only using random UUIDs, you need to worry about this, because UUIDs always contain a version and variant number.
> Now, you can print UUIDs any way you like. And if you print them different places with different rendering libraries they can look different, sure. Don't confuse that with the underlying value, which is a random string.
The string is only mostly random. It contains version information.
Huh? Strings are stored and transmitted as bytes, there is no need to interpret the subfields as any given endianness if the whole thing is stored as 16 bytes.
Alternatively you can interpret the hexadecimal encoding of the subfields to raw values, and compact things into a single 16 byte / 128-bit value, which I'm assuming is what's being referred to by "storing UUIDs efficiently as bytes".
To perform this compacting requires some interpretation - if only from hexadecimal to bytes. In implementing this, you might feed the entire first subfield to your hex parsing function. The result, however, will be an integer with more bits than a byte, at which point you need to care about endianness.
You can choose to instead parse two hex characters as a time and treat each as a byte. You could argue that this is "not" interpreting the subfield as any given endianness - but this is semantically identical to interpreting the field as big endian, so I'd argue that you are.
Issues that don't exist if you don't use a hideously complex format like UUID in the first place.
> Yes there are many bad ways to generate them, but your library should offer the right way to generate them.
Yes, a UUID library will have a way to generate the random kind of UUID, but it's telling that you need a library for it.
You can generate 16 random bytes without a library, by using fopen() and fread().
> Yes the string format is weird, but why would you write that by hand? Let the library print it, let the library parse it.
You shouldn't need a library to handle your identifiers. If you do, your identifiers are too complicated.
> Want to store it in a database? There's a column type for that. Want to send it over the network? There's probably a datatype for it in whatever protocol you're using (whereas with your approach you immediately have endianness issues).
With the approach of random bytes, there's at least a consistent endianness throughout. And you can solve the endianness issue by treating it as a byte string. Job done.
Sounds more complicated and easier to get wrong. I'd rather have a single function call with an obvious name.
Yeah but then good luck getting the right /dev/random vs /dev/urandom on the right platform, and also good luck getting it to work on windows and ios.
Although if you were using GUIDs in indexes in SQL Server, it made sense for them to have some similarity and allow ordering [1]
Last time I benchmarked this, however, I couldn't detect any difference.
[1] http://blogs.msdn.com/b/sqlserverfaq/archive/2010/05/27/guid...
MD5 and SHA-1 being broken only matter if you let users choose your keys, to which the answer is don't do that.
Another thing to consider, is that due to the birthday paradox once you build 2.7 * 10^18 GUID, the probability that you have at lest a collision is bigger than 50%. And 2.7 * 10^18 is only 2^61.2.
In any case, I should have been more clear. I'm not necessarily worried about this case. But if you design systems and miss a factor of a square root, you tend to break things that we're all relying on to work.
It's going to be a while before the birthday paradox is really going to be problematic.
There may be duplicate GUID:s used in different contexts and that's just fine. If there is single system and context where storing just GUID:s takes several exabytes we may start worrying.
Be careful there. Parts of Europe including here (the UK) officially use short scale like the US.
When looking at historical figures it is important to be extra careful as scale use has flipped back and forth over time in places. "Historical" doesn't go as far back as you think either: short scale becoming the standard number naming convention in the UK happened in 1974 so there are still people alive who use long scale and remember it being the most common form.
To really confuse things some places use a half-way house of "short scale with milliard"...
It is safer to stick with "scientific" prefixes (kilo, mega, giga, tera, ...) as they are consistently interpreted the same way except where someone is being deliberately difficult (for "deliberately difficult" read "just plain wrong"). It sometimes sounds odd referring to things like "giga pounds" instead of "billions of pounds" (or "thousand millions of pounds" or "milliards of pounds") but it reduces the risk of misinterpretation and anyone who doesn't understand probably wouldn't truly understand any of the above terms without explanation.
For more see https://en.wikipedia.org/wiki/Long_and_short_scales