Why are we celebrating proprietary, vendor-specific-extensions in our open source tools that are, by nature, supposed to operate in a vendor-agnostic manner? This flag makes zero sense for anyone who isn't an AWS customer. This is even more insulting considering that "being an AWS customer" is not free (both as in beer and freedom). It really surprises me that this made it into curl.
I've always been a fan of Terraform's provider architecture--all the vendor-specific bits are implemented as third-party plugins that implement a standard API and communicate with Terraform itself in a client-server model.
The AWS v4 signing method is a spec for how to do HMAC request signing. It is not limited to use only with AWS services.
You can use it to sign requests on any service, and there are many implementations. Here is an example of one I’ve used in the past: https://github.com/auroratechnologies/vangoh
Isn't that the beauty of open source? That we are empowered to make tools that fit our needs and the needs of our users?
I don't see why open source tools should be intentionally crippled in the name of some sort of "purity". Implementing this feature costs nothing (it is an open standard, unencumbered by licenses or patents), and you can continue to use the tool as if the vendor-specific option doesn't exist.
It's less about purity and more about overhead. Adding this feature adds friction to refactoring the code, adds surface area for vulnerabilities (and the user ignoring it won't make that go away), and makes the binary bigger. The cost/benefit ratio might well be favorable, but implementing this feature absolutely costs something.
> celebrating proprietary, vendor-specific-extensions in our open source tools
It's not proprietary nor vendor specific. I'm not sure what makes cURL "our" tool?
> by nature, supposed to operate in a vendor-agnostic manner?
Does the inclusion of this feature mean a different feature cannot coexist with it? Was it so absurdly complicated to develop that it took time away from other integrations?
> This flag makes zero sense for anyone who isn't an AWS customer.
Do you also dislike QUIC or RTMP? How about NTLM?
> that "being an AWS customer" is not free
Yea but "using AWS protocols outside of AWS" _is_ free.
> It really surprises me that this made it into curl.
They should turn down useful but impure contributions? The lead developer of cURL still has a day job.
Because some people believe tools exist to make lives easier, not to evangelize some sort of ideology. A lot of people use AWS; adding helpers for it into commonly-used tools can make people's lives easier.
And beyond that, there a bunch of clones of various AWS APIs, both in other proprietary products, and in open source alternatives. I bet this functionality works with those, too.
It's quite a clean extension that can be disabled at compile time. It doesn't complicate the rest of curl, and provides a useful way to invoke any AWS API from the CLI without needing a giant AWS library dependency.
As an open source maintainer, this is what I would consider a slippery slope. Sure, this one is well-specified and "not that complex" until N years later there are 30 compile-time flags for all sorts of vendor-specific features and a new CVE every week from the massive increase in highly-use-case-specific code to maintain.
AWS uses one algorithm to authenticate requests against all of its APIs. Many of its APIs accept simple query parameters or JSON, making them easy to call with curl.
I appreciate the shout! To accommodate the influx of people coming to the blog, I updated the post to include IAM role assumption credentials as well. I hope you all enjoy!
Just mentioning that I love AWS' signed request model. It's so much easier than dealing with OAuth for APIs and I wish it would catch on more broadly.
It's inspired an IETF RFC (https://datatracker.ietf.org/doc/html/rfc9421), so we'll see.
17 comments
[ 5.2 ms ] story [ 60.1 ms ] threadOr for calling AWS APIs in containers, etc. without installing heavyweight AWS CLI.
I've always been a fan of Terraform's provider architecture--all the vendor-specific bits are implemented as third-party plugins that implement a standard API and communicate with Terraform itself in a client-server model.
The AWS v4 signing method is a spec for how to do HMAC request signing. It is not limited to use only with AWS services.
You can use it to sign requests on any service, and there are many implementations. Here is an example of one I’ve used in the past: https://github.com/auroratechnologies/vangoh
Or as mentioned above: https://github.com/minio/minio/blob/master/cmd/signature-v4....
I don't see why open source tools should be intentionally crippled in the name of some sort of "purity". Implementing this feature costs nothing (it is an open standard, unencumbered by licenses or patents), and you can continue to use the tool as if the vendor-specific option doesn't exist.
It's not proprietary nor vendor specific. I'm not sure what makes cURL "our" tool?
> by nature, supposed to operate in a vendor-agnostic manner?
Does the inclusion of this feature mean a different feature cannot coexist with it? Was it so absurdly complicated to develop that it took time away from other integrations?
> This flag makes zero sense for anyone who isn't an AWS customer.
Do you also dislike QUIC or RTMP? How about NTLM?
> that "being an AWS customer" is not free
Yea but "using AWS protocols outside of AWS" _is_ free.
> It really surprises me that this made it into curl.
They should turn down useful but impure contributions? The lead developer of cURL still has a day job.
And beyond that, there a bunch of clones of various AWS APIs, both in other proprietary products, and in open source alternatives. I bet this functionality works with those, too.
It's well worth a thousand lines of code.
https://github.com/curl/curl/blob/b9fea261e632b9493ea6038bfb...
https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_a...
You can implement quite easily in any language without touching any Amazon tooling or infrastructure.