I used that imagery mostly because other preceding resources about Macaroons use macarons (there's even an unofficial logo[0] for macaroons that are actually macarons). But according to wikipedia[1], macarons are simply the French variety of macaroons.
If you ask me, French macarons are way better than the coconut kind.
Macaroons can be attenuated and delegated in ways that JWTs can not. Restricting the access granted by a JWT requires having the initial keys used to sign the token. Macaroons also have a built-in mechanism for verifying assertions made by third parties (in the form of "third party caveats")
So you have a list of constraints: c1, c2, c3. You have a secret key on your server, 'k'. You calculate the signature as 'HMAC( HMAC( HMAC(k, c1), c2) c3)'.
So anyone can add constraints and then create a new valid signature by HMAC'ing the additional constraint into the existing signature, right? You can add constraints, but of course, you can't remove them later if you only have the latest iteration of the signature.
I'm trying to see the use-case here... is it a 3rd party taking one of these signed strings in order to append their own "constraints" onto it, cycle the signature, pass it to an end-user, who will then hand it back to your server? And even if this happened, how would you know at the end of the day who actually added the constraint? All but the first HMAC can be done by anyone, so aren't you just creating work for yourself?
Specifically how is this ever better than just doing a single HMAC(key, data) on everything that you control, specifying some delimiter character to know where your own signed data ends, and then everything else is just unsigned? I guess if you trust your link to the 3rd party, and if you trust they will add the proper constraints, then the only benefit is you know once constraints are added they cannot be removed without access to the original signature.
Going directly to the paper, they show an example on Page 4 (http://imgur.com/ARJNdu6) of a 3rd party service adding additional constraints onto an existing set;
For example, Figure 4 shows a macaroon authorizing a user at client C to
access chunk 235 at a key-value storage service TS. This macaroon has
been derived as a strict attenuation of another, more permissive macaroon
—held by the social-networking “forum service” FS—by both eliding that
macaroon’s signature (called, say KFS), and by adding caveats whose
predicates require a specific chunk and client IP address to be present
in the request context. Even so, this macaroon’s signature, KUSER, both
allows the target storage service to verify its integrity, and also
allows its bearer to derive new, further attenuated macaroons. In this
manner, macaroon signatures both protect integrity and act as keys.
"Integrity" usually means you know something was not altered, so I think it's a bit dangerous to call it that. The coloring of the figure is also misleading. With the proposed nested HMAC construction, we do not actually know which caveats were added by TS versus which were added by FS, or perhaps Mallory. All that 'TS' knows is that these constraints exist, it loses a key piece of information, namely, the ability to prove which of the constraints are guaranteed to be first-party constraints, versus which were added later by third parties. For example, this could expose 'TS' to potential DoS attacks because anyone can add potentially expensive-to-verify "first-party" constraints which the server could be tricked into thinking that it had actually added itself.
Instead, the server should HMAC the full set of its own constraints with its own key, and then a single delimiter should be added. If third-parties want to add additional sets of constraints, then the nested HMAC can start being used, but again I would propose, first, sets of constraints be added atomically with some delimiter to reduce the number of wasted HMAC cycles, and second, there should be a way to include an additional shared secret between TS and FS in the HMAC process so that TS can actually validate who added the constraints without significantly increasing the size of the payload.
Edit: So, what I get out of this after ~30 minutes of review is it's a very high level protocol for how an already trusted set of servers might go about letting each other specify constraints. Frankly, I'm not sure why that's 16 pages of research. I think the HMAC cha...
Edit: responding to you is a bit of a moving target. I'll attempt to address your latest revision a bit later.
Most of what this article is about concerns first-party caveats only, because (for better or worse) I was trying to keep the discussion fairly basic. I'm glad it piqued your interest enough to read the macaroon paper!
> Instead, the server should HMAC the full set of its own constraints with its own key, and then a single delimiter should be added.
This basically describes a JWT, and there's nothing wrong with doing things that way; it clearly works well for a lot of cases. Macaroons provide flexibility by allowing attenuation and delegation. Hyperdex[0] and this video[1] both discuss cases where those properties may be more desirable, which may serve as clarifying examples.
> I would propose sets of constraints be added atomically with some delimiter to reduce the number of wasted HMAC cycles.
That's perfectly valid, it's also a suggestion from the macaroon paper:
> Macaroons permit many optimizations, in both their primitives and applications. For example, a reduced number of operations can be used for chained-MACs, if caveats are combined as in Figure 9.
Where figure 9 just shows including multiple "caveat assertions" in a single "caveat".
> This exposes 'TS' to potential DoS attacks where anyone can add potentially expensive-to-verify first-party constraints which there server could be tricked into thinking that it had actually added itself.
It's good to point out that this is a vector, but practically it's very easy to mitigate this sort of attack. The entire verification process is controlled by TS, so in the first place any caveats it doesn't understand will be ignored (and verification will fail because not all constraints were met). I strongly prefer failing macaroon verification as soon as possible (even if it opens up the possibility of certain timing attacks; which could at best only reveal supported caveat types and never any secrets). This alone limits the attack surface only to specific caveat formats that TS understands. From there it's easy to add other safeguards if necessary. Verification could limit the number of caveats allowed (the macaroon paper even suggests these limits could be encoded in the macaroon as "structural caveats", although currently no implementations support them), or simply fail if it takes too long to verify. As with all DoS attack prevention the goal is to mitigate as much as possible; simply rate limiting could be effective.
> I guess if you trust your link to the 3rd party, and you trust they will add the proper constraints, the only benefit is you know once they are added they cannot be removed.
When you use "third-party" you seem to be referring to attenuating the macaroon with additional "first-party" caveats. Actual "third-party" caveats can be used to verify context in a third-party system. First-party caveats confirm the context of the target service, but consider that first-party caveats could be added by other components of TS (e.g. in a service-oriented architecture) or by local client software (e.g. a browser extension), not just "third parties."
> We do now actually know which caveats were added by TS versus which were added by FS.
This is not quite true - without storing information about the originally constructed macaroon, TS has no way of knowing which caveats it issued originally and which caveats were appended later by someone else.
Thanks for the detailed response, and sorry for all the ninja-edits, I kept coming back and adding more as I read more!
Please note, "We do not actually know..." -- I agree with you emphatically. I think it's my biggest gripe about the construct.
I do like the idea of a standard way of packaging a set of constraints, and also that they can be flexibly added by different systems. I think there's some weakness in how they are signed and to describe the signature as providing 'integrity' is over-reaching.
I might call it an "append-only cookie" or an "extendable cookie" but never a "macaroon". ;-)
They don't out of the box form an ocap system but there's nothing preventing you from making one with them. That's something I've been meaning to explore more in-depth.
Awesome! Thanks. I've read the Macaroons research paper now and of course it mentions "prior work going back to the Amoeba OS" and they thank Mark Miller (among others) at the end, so yeah.. I feel a little sheepish. :-)
This is very exciting stuff. Thank you for writing such a great article about it, I'm looking forward to reading more. And I have a copy of your pymacaroons source printed out to peruse. Cheers!
10 comments
[ 2.6 ms ] story [ 46.7 ms ] threadIf you ask me, French macarons are way better than the coconut kind.
[0]: http://macaroons.io/
[1]: https://en.wikipedia.org/wiki/Macaroon#France
JWT is mentioned in passing near the end of the article, but it would be great to see a description of what makes macaroons different or better.
So anyone can add constraints and then create a new valid signature by HMAC'ing the additional constraint into the existing signature, right? You can add constraints, but of course, you can't remove them later if you only have the latest iteration of the signature.
I'm trying to see the use-case here... is it a 3rd party taking one of these signed strings in order to append their own "constraints" onto it, cycle the signature, pass it to an end-user, who will then hand it back to your server? And even if this happened, how would you know at the end of the day who actually added the constraint? All but the first HMAC can be done by anyone, so aren't you just creating work for yourself?
Specifically how is this ever better than just doing a single HMAC(key, data) on everything that you control, specifying some delimiter character to know where your own signed data ends, and then everything else is just unsigned? I guess if you trust your link to the 3rd party, and if you trust they will add the proper constraints, then the only benefit is you know once constraints are added they cannot be removed without access to the original signature.
Going directly to the paper, they show an example on Page 4 (http://imgur.com/ARJNdu6) of a 3rd party service adding additional constraints onto an existing set;
"Integrity" usually means you know something was not altered, so I think it's a bit dangerous to call it that. The coloring of the figure is also misleading. With the proposed nested HMAC construction, we do not actually know which caveats were added by TS versus which were added by FS, or perhaps Mallory. All that 'TS' knows is that these constraints exist, it loses a key piece of information, namely, the ability to prove which of the constraints are guaranteed to be first-party constraints, versus which were added later by third parties. For example, this could expose 'TS' to potential DoS attacks because anyone can add potentially expensive-to-verify "first-party" constraints which the server could be tricked into thinking that it had actually added itself.Instead, the server should HMAC the full set of its own constraints with its own key, and then a single delimiter should be added. If third-parties want to add additional sets of constraints, then the nested HMAC can start being used, but again I would propose, first, sets of constraints be added atomically with some delimiter to reduce the number of wasted HMAC cycles, and second, there should be a way to include an additional shared secret between TS and FS in the HMAC process so that TS can actually validate who added the constraints without significantly increasing the size of the payload.
Edit: So, what I get out of this after ~30 minutes of review is it's a very high level protocol for how an already trusted set of servers might go about letting each other specify constraints. Frankly, I'm not sure why that's 16 pages of research. I think the HMAC cha...
Most of what this article is about concerns first-party caveats only, because (for better or worse) I was trying to keep the discussion fairly basic. I'm glad it piqued your interest enough to read the macaroon paper!
> Instead, the server should HMAC the full set of its own constraints with its own key, and then a single delimiter should be added.
This basically describes a JWT, and there's nothing wrong with doing things that way; it clearly works well for a lot of cases. Macaroons provide flexibility by allowing attenuation and delegation. Hyperdex[0] and this video[1] both discuss cases where those properties may be more desirable, which may serve as clarifying examples.
> I would propose sets of constraints be added atomically with some delimiter to reduce the number of wasted HMAC cycles.
That's perfectly valid, it's also a suggestion from the macaroon paper:
> Macaroons permit many optimizations, in both their primitives and applications. For example, a reduced number of operations can be used for chained-MACs, if caveats are combined as in Figure 9.
Where figure 9 just shows including multiple "caveat assertions" in a single "caveat".
> This exposes 'TS' to potential DoS attacks where anyone can add potentially expensive-to-verify first-party constraints which there server could be tricked into thinking that it had actually added itself.
It's good to point out that this is a vector, but practically it's very easy to mitigate this sort of attack. The entire verification process is controlled by TS, so in the first place any caveats it doesn't understand will be ignored (and verification will fail because not all constraints were met). I strongly prefer failing macaroon verification as soon as possible (even if it opens up the possibility of certain timing attacks; which could at best only reveal supported caveat types and never any secrets). This alone limits the attack surface only to specific caveat formats that TS understands. From there it's easy to add other safeguards if necessary. Verification could limit the number of caveats allowed (the macaroon paper even suggests these limits could be encoded in the macaroon as "structural caveats", although currently no implementations support them), or simply fail if it takes too long to verify. As with all DoS attack prevention the goal is to mitigate as much as possible; simply rate limiting could be effective.
> I guess if you trust your link to the 3rd party, and you trust they will add the proper constraints, the only benefit is you know once they are added they cannot be removed.
When you use "third-party" you seem to be referring to attenuating the macaroon with additional "first-party" caveats. Actual "third-party" caveats can be used to verify context in a third-party system. First-party caveats confirm the context of the target service, but consider that first-party caveats could be added by other components of TS (e.g. in a service-oriented architecture) or by local client software (e.g. a browser extension), not just "third parties."
> We do now actually know which caveats were added by TS versus which were added by FS.
This is not quite true - without storing information about the originally constructed macaroon, TS has no way of knowing which caveats it issued originally and which caveats were appended later by someone else.
[0]: http://hyperdex.org/doc/latest/Authorization/#x11-630002 [1]: https://air.mozilla.org/macaroons-cookies-with-contextual-ca...
Please note, "We do not actually know..." -- I agree with you emphatically. I think it's my biggest gripe about the construct.
I do like the idea of a standard way of packaging a set of constraints, and also that they can be flexibly added by different systems. I think there's some weakness in how they are signed and to describe the signature as providing 'integrity' is over-reaching.
I might call it an "append-only cookie" or an "extendable cookie" but never a "macaroon". ;-)
Here's a good discussion on the differences between macaroons and capabilities: http://www.eros-os.org/pipermail/cap-talk/2014-December/0163... (see the "Macaroons: Capabilities vs Credentials).
This is very exciting stuff. Thank you for writing such a great article about it, I'm looking forward to reading more. And I have a copy of your pymacaroons source printed out to peruse. Cheers!