35 comments

[ 3.0 ms ] story [ 44.8 ms ] thread
This is a link to just one section of a larger book. The next section compare CBOR with a number of other binary storage format, such as protobuf.
Odd that the XML and JSON sections show examples of the format, but CBOR doesn’t. I’m left with no idea what it looks like, other than “building on JSON’s key/value format”.
How different is CBOR compared to BSON? Both seem to be binary json-like representations.

Edit: BSON seems to contain more data types than JSON, and as such it is more complex, whereas CBOR doesn't add to JSON's existing structure.

people are just straight up afraid to write their own binary formats, aren't they.

it's not hard, it's exactly like creating your own text format but you write binary data instead of text, and you can't read it with your eyes right away (but you can after you've looked at enough of it.) there is nothing to fear or to even worry about; just try it. look up how things like TLV work on wikipedia. you can do just about anything you would ever need with plain binary TLV and it's gonna perform like you wouldn't believe.

https://en.wikipedia.org/wiki/Type%E2%80%93length%E2%80%93va...

binary formats are always going to be 1-2 orders of magnitude faster than plain text formats, no matter which plain text format you're using. writing a viewer so you can easily read the data isn't zero-effort like it is for JSON or XML where any existing text editor will do, but it's not exactly hard, either. your binary format reading code is the core of what that viewer would be.

once you write and use your own binary format, existing binary formats you come across become a lot less opaque, and it starts to feel like you're developing a mild superpower.

CBOR has always seemed to me like the most promising data format for efficient data transfer. Somewhat weird how little use it has.
CBOR is when you need option for very small code size. If you can always use compression, CBOR provides no significant data size improvement over JSON.

With small code size it beats also BSON, EBML and others.

I wish browsers would support CBOR natively so I could just return CBOR instead of JSON(++speed --size ==win) and not have to be concerned with decoding it or not being able to debug requests in dev console.
ASN.1 while complex has really seems to be a step up from those (even if older) in terms of terseness (as binary encoding) and generality.
The FOSS tooling for it sucks balls. That's why
Would you rather write a parser for this:

    SEQUENCE {
      SEQUENCE {
        OBJECT IDENTIFIER '1 2 840 113549 1 1 1'
        NULL
        }
      BIT STRING 0 unused bits, encapsulates {
          SEQUENCE {
            INTEGER
              00 EB 11 E7 B4 46 2E 09 BB 3F 90 7E 25 98 BA 2F
              C4 F5 41 92 5D AB BF D8 FF 0B 8E 74 C3 F1 5E 14
              9E 7F B6 14 06 55 18 4D E4 2F 6D DB CD EA 14 2D
              8B F8 3D E9 5E 07 78 1F 98 98 83 24 E2 94 DC DB
              39 2F 82 89 01 45 07 8C 5C 03 79 BB 74 34 FF AC
              04 AD 15 29 E4 C0 4C BD 98 AF F4 B7 6D 3F F1 87
              2F B5 C6 D8 F8 46 47 55 ED F5 71 4E 7E 7A 2D BE
              2E 75 49 F0 BB 12 B8 57 96 F9 3D D3 8A 8F FF 97
              73
            INTEGER 65537
            }
          }
      }
or this:

    (public-key
      (rsa
        (e 65537)
        (n
         165071726774300746220448927123206364028774814791758998398858897954156302007761692873754545479643969345816518330759318956949640997453881810518810470402537189804357876129675511237354284731082047260695951082386841026898616038200651610616199959087780217655249147161066729973643243611871694748249209548180369151859)))
I know that I’d prefer the latter. Yes, we could debate whether the big integer should be a Base64-encoded binary integer or not, but regardless writing a parser for the former is significantly more work.

And let’s not even get started with DER/BER/PEM and all that insanity. Just give me text!

The article reads like a semi-slop with its numerous lists and overly long explanations of obvious things, such as how XML came to be.
The only mention I can see in this document of compression is

> Significantly smaller than JSON without complex compression

Although compression of JSON could be considered complex, it's also extremely simple in that it's widely used and usually performed in a distinct step - often transparently to a user. Gzip, and increasingly zstd are widely used.

I'd be interested to see a comparison between compressed JSON and CBOR, I'm quite surprised that this hasn't been included.

Love or hate JSON, the beauty and utility stem from the fact that you have only the fundamental datatypes as a requirement, and that's it.

Structured data that, by nesting, pleases the human eye, reduced to the max in a key-value fashion, pure minimalism.

And while you have to write type converters all the time for datetime, BLOBs etc., these converters are the real reasons why JSON is so useful: every OS or framework provides the heavy lifting for it.

So any elaborated new silver bullet would require solving the converter/mapper problem, which it can't.

And you can complain or explain with JSON: "Comments not a feature?! WTF!" - Add a field with the key "comment"

Some smart guys went the extra mile and nevertheless demanded more, because wouldn't it be nice to have some sort of "strict JSON"? JSON schema was born.

And here you can visibly experience the inner conflict of "on the one hand" vs "on the other hand". Applying schemas to JSON is a good cause and reasonable, but guess what happens to JSON? It looks like unreadable bloat, which means XML.

Extensibility is fine, basic operations appeal to both demands, simple and sophisticated, and don't impose the sophistication on you just for a simple 3-field exchange about dog food preferences.

CBOR (and MsgPack) still embraces that simplicity. It provides the same types of key-value, lists, and basic values.

However the types are more precise allowing you to differentiate between int32’s or int64’s or between strings or bytes.

Essentially you can replace json with it and gain performance, less ambiguity but with the same flexibility. You do need a step to print CBOR in human readable form, but it has a standardized human readable form similar to a typed json.

> you have only the fundamental datatypes as a requirement

Not really; the set of datatypes has problems. It uses Unicode, not binary data and not non-Unicode text. Numbers are usually interpreted as floating point numbers rather than integers, which can also be a problem. Keys can only be strings. And, other problems. So, the data types are not very good.

And, since it is a text format, it means that escaping is required.

> And while you have to write type converters all the time for datetime, BLOBs etc.

Not having a proper data type for binary means that you will need to encode it using different types and then avoids the benefit of JSON, anyways. So, I think JSON is not as helpful.

I think DER is better (you do not have to use all of the types; only the types that you are using is necessary to be implemented, because the format of DER makes it possible to skip anything that you do not care about), and I made up TER which is text based format which can be converted to DER (so, even though a binary data is represented as text, it is still representing the binary data type, rather than needing to use the wrong data type like JSON does).

> And you can complain or explain with JSON: "Comments not a feature?! WTF!" - Add a field with the key "comment"

But then it is a part of the data, which you might not want.

Oh good, another CBOR thread. Disclaimer: I wrote and maintain a MessagePack implementation. I've also bird dogged this for a while, HN search me.

Mostly, I just want to offer a gentle critique of this book's comparison with MessagePack [0].

> Encoding Details: CBOR supports indefinite-length arrays and maps (beneficial for streaming when total size is unknown), while MessagePack typically requires fixed collection counts.

This refers to CBOR's indefinite length types, but awkwardly, streaming is a protocol level feature, not a data format level feature. As a result, there's many better options, ranging from "use HTTP" to "simply send more than 1 message". Crucially, CBOR provides no facility for re-syncing a stream in the event of an error, whether that's network or simply a bad encoding. "More features" is not necessarily better.

> Standardization: CBOR is a formal IETF standard (RFC 8949) developed through consensus, whereas MessagePack uses a community-maintained specification. Many view CBOR as a more rigorous standard inspired by MessagePack.

Well, CBOR is MessagePack. Carsten Bormann forked MessagePack, changed some of the tag values, wrote a standard around it, and submitted it to the IETF against the wishes of MessagePack's creators.

> Extensibility: CBOR employs a standardized semantic tag system with an IANA registry for extended types (dates, URIs, bignums). MessagePack uses a simpler but less structured ext type where applications define tag meanings.

Warning: I have a big rant about the tag registry.

The facilities are the same (well, the tag is 8 bytes instead of 1 byte, but w/e); it's TLV all the way down (Bormann ripped this also). Bormann's contribution is the registry, which is bonkers [1]. There's... dozens of extensions there? Hundreds? No CBOR implementation supports anywhere near all this stuff. "Universal Geographical Area Description (GAD) description of velocity"? "ur:request, Transaction Request identifier"?

The registry isn't useful. Here are the possible scenarios:

If something is in high demand and has good support across platforms, then it's a no-brainer to reserve a tag. MP does this with timestamps.

If something is in high demand, but doesn't have good support across platforms, then you're putting extra burden on those platforms. Ex: it's not great if my tiny microcontroller now has to support bignums or 128-bit UUIDs. Maybe you do that, or you make them optional, but that leads us to...

If something isn't in high demand or can't easily be supported across platforms, but you want support for it anyway, there's no need to tell anyone else you're using that thing. You can just use it. That's MP's ext types.

CBOR seems to imagine that there's a hypothetical general purpose decoder out there that you can point to any CBOR API, but there isn't and there never will be. Nothing will support both "Used to mark pointers in PSA Crypto API IPC implementation" and "PlatformV_HAS_PROPERTY" (I just cannot get over this stuff). There is no world where you tell the IETF about your tags, define an API with them, and someone completely independently builds a decoder for them. It will always be a person who cares about your specific tags, in which case, why not just agree on the ext types ahead of time? A COSE decoder doesn't need also need to decode a "RAINS Message".

> Performance and Size: Comparisons vary by implementation and data. CBOR prioritizes small codec size (for constrained devices) alongside message compactness, while MessagePack focuses primarily on message size and speed.

I can't say I fully understand what this means, but CBOR and MP are equivalent here, because CBOR is MP.

> Conceptual Simplicity: MessagePack's shorter specification appears simpler, but CBOR's unification of types under its major type/additional info ...

> Well, CBOR is MessagePack. Carsten Bormann forked MessagePack

Sure, that’s sort of true but missing context. Bormann (and others) wanted to add things such as separate string and byte sequence types. The MessagePack creator refused for years. Fair enough it’s his format. But it frustrated the community dealing with string vs bytes issues. It also highlights a core philosophical difference of a mostly closed spec vs an extensible first one.

> changed some of the tag values, wrote a standard around it, and submitted it to the IETF against the wishes of MessagePack's creators.

That’s just incorrect and a childish way to view it in my opinion.

The core philosophy and mental models are different in key aspects.

MessagePack is designed as a small self mostly closed format. It uses a simple TLV format with a couple hundred possible user extensions and some clever optimizations. The MP “spec” focuses on this.

CBOR re-envisioned the core idea of MessagePack from the ground up as an extensible major/minor tag system. It’s debatable how much CBOR is a fork of MPack vs a new format with similarities.

The resulting binary output is pretty similar with similar benefits but the core theoretical models are pretty different. The IETF standard bares little to no resemblance to the MessagePack specification.

> The facilities are the same (well, the tag is 8 bytes instead of 1 byte, but w/e); it's TLV all the way down (Bormann ripped this also).

The whole point of CBOR is that the tags go from 1-8 bytes. The parser designs end up fairly different due to the different tag formats. I’ve written and ported parsers for both.

It’s not like the MessagePack creator invented TLV formats either. He just created an efficient and elegant one that’s pretty general. No one says he ripped off “TLV”.

You can’t just take a message pack parser and turn it into a CBOR one by changing some values. I’ve tried and it turns out poorly and doesn't support much of CBOR.

> This refers to CBOR's indefinite length types, but awkwardly, streaming is a protocol level feature, not a data format level feature.

The indefinite length format is very useful for embedded space. I’ve hit limits with MessagePack before on embedded projects because you need to know the length of an array upfront. I wished I’d had CBOR instead.

This can also be useful for data processing applications. For example streaming the conversion of a large XML file into a more concise CBOR format would be much more memory efficient. For large scale that’s pretty handy.

> > However, MessagePack sacrifices human-readability > This, of course, applies to CBOR as well.

For the binary format yes. However the CBOR specification defines an official human readable text format for debugging and documentation purposes. It also defines a schema system like json-schema but for CBOR.

Turns out “just some specs” can actually be pretty valuable.

> This refers to CBOR's indefinite length types, but awkwardly, streaming is a protocol level feature, not a data format level feature.

BER also has indefinite length as well as definite length, but the way that it is doing, is not very good (DER only uses definite length). I think it is more helpful to use a different format when streaming with indefinite length is require, so I made up DSER (and SDSER) which is working as follows:

- The type, which is encoded same as DER.

- If it is constructed, all items it contains come next (the length is omitted).

- If it is primitive, zero or more segments, each of which starts with one byte in range 0x01 to 0xFF telling how many bytes of data are in that segment. (The value is then just the concatenation of all segments together.)

- For both primitive and constructed, one byte with value 0x00 is the termination code.

> Bormann's contribution is the registry, which is bonkers [1]. There's... dozens of extensions there? Hundreds? No CBOR implementation supports anywhere near all this stuff.

It should not need to support all of that stuff; you will only use the ones that are relevant for your program. (There is also the similar kind of complaint with ASN.1, and the similar response that I had made.)

> If something is in high demand, but doesn't have good support across platforms, then you're putting extra burden on those platforms. Ex: it's not great if my tiny microcontroller now has to support bignums or 128-bit UUIDs.

Although it is a valid concern, you would use data which does not have numbers bigger than you need to be, so it can avoid such a problem. You can treat UUIDs like octet strings, although if you only need small numbers then you should use the small numbers types instead, anyways.

> If something isn't in high demand or can't easily be supported across platforms, but you want support for it anyway, there's no need to tell anyone else you're using that thing.

Sometimes it is useful to tell someone else that you are using that thing, although often it is unnecessary, like you said.

Erlang / Elixir has amazing support for ASN.1! I love it.

https://www.erlang.org/doc/apps/asn1/asn1_getting_started.ht...

https://www2.erlang.org/documentation/doc-14/lib/asn1-5.1/do... (https://www2.erlang.org/documentation/doc-14/lib/asn1-5.1/do...)

I am using ASN.1 to communicate between a client (Java / Kotlin) and server (Erlang / Elixir), but unfortunately Java / Kotlin has somewhat of a shitty support for ASN.1 in comparison to Erlang.

Erlang and ASN.1 are from telecom, so it makes sense they have best support
I also use ASN.1 but I use C, so I wrote my own implementation of DER.
I admit I got nerd-sniped here, but the table for floats[1] suggests that 10000.0 be represented as a float32. However, isn't it exactly representable as 0x70e2 in float16[2]? There are only 10 significant bits to the mantissa (including the implicit 1), while float16 has 11 so there's even an extra bit to spare.

1: https://cborbook.com/part_1/practical_introduction_to_cbor.h...

2: i.e. 1.220703125×2¹³

Looks like it's a typo; they state:

> 0x47c35000 encodes 10000.0

But by my math that encodes 100000.0 (note the extra zero).

Related. Others?

Begrudgingly Choosing CBOR over MessagePack - https://news.ycombinator.com/item?id=43229259 - March 2025 (78 comments)

MessagePack vs. CBOR (RFC7049) - https://news.ycombinator.com/item?id=23838565 - July 2020 (2 comments)

CBOR – Concise Binary Object Representation - https://news.ycombinator.com/item?id=20603378 - Aug 2019 (71 comments)

CBOR – Concise Binary Object Representation - https://news.ycombinator.com/item?id=10995726 - Jan 2016 (36 comments)

Libcbor – CBOR implementation for C and others - https://news.ycombinator.com/item?id=9597198 - May 2015 (5 comments)

CBOR – A new object encoding format - https://news.ycombinator.com/item?id=6932089 - Dec 2013 (9 comments)

RFC 7049 - Concise Binary Object Representation (CBOR) - https://news.ycombinator.com/item?id=6632576 - Oct 2013 (52 comments)

Fun fact: CBOR is used within the WebAuthn (Passkey) protocol.

To do Passkey-verification server-side, I had to implement a pure-SQL/PLpgSQL CBOR parser, out of fear that a C-implementation could crash the PostgreSQL server: https://github.com/truthly/pg-cbor

And .Net 5 circa 2020 added support for CBOR. ASP.NET ended up being a good choice for an experimental WebAuthn server for FedCM and DID experiments.
If GML was an infant, SGML is the bright youngster far exceeds expectations and made its parents too proud, but XML is the drug-addicted gang member who had committed his first murder before he had sex, which was rape.

https://www.schnada.de/grapt/eriknaggum-xmlrant.html

We're going to have to think up something worse for CBOR.

Yeah, CBOR is very good, but you will only understand after you digest 100000 words about it :D.
I prefer DER, which is also a binary format so it has the advantages of binary formats, too. (There is also BER, but in my opinion, DER is better.) I use DER in some programs, if the structured data format is useful. (Also, since text format is sometimes useful too, I had made up TER which is intended to be converted to DER. The DER file can be made in other ways as well and it is not required to use TER.)

(Also, standard ASN.1 does not have a key/value list type (which JSON and CBOR do have), but I had made up some nonstandard extensions to ASN.1 (called ASN.1X), including a few additional types, one of which is the key/value list type. Due to this, ASN.1X can now make a superset of the data that can be made by JSON (the only new type that is needed for this is the key/value list type; the other types of JSON are already standard ASN.1 types).)