Does anyone else finds AWS and other Amazon services overly complicated?
1. you request client credentials, which is normal.
2. construct request URL, normal.
3. add headers, eh, normal.
4. signature... fuck.
First you need to convert the URL you have from step 2, mash it with headers from step 3, add header keys to signed headers, then sum256 hash the payload and hex encode it.
Then you create a sign, add algorithm, request date time that is formatted with ISO8601 but all special characters stripped apart, add credential scopes, hash the canonical request you created at the first step.
Then, you calculate this abomination: HMAC(HMAC(HMAC(HMAC("AWS4" + kSecret,"20150830"),"us-east-1"),"iam"),"aws4_request")
after that you calculate this: signature = HexEncode(HMAC(derived signing key, string to sign))
after that you create an authorization header and add signature to it: Authorization: AWS4-HMAC-SHA256 Credential=AKIAIHV6HIXXXXXXX/20201022/us-east-1/execute-api/aws4_request, SignedHeaders=host;user-agent;x-amz-access-token;x-amz-date, Signature=5d672d79c15b13162d9279b0855cfba6789a8edb4c82c400e06b5924aEXAMPLE
...I mean what the fuck? I can understand why people choose Azure over AWS for the sake of freaking simplicity just by looking at this sign and request process. It feels overly-complicated. Does anyone feels the same while working with this abomination?
147 comments
[ 3.4 ms ] story [ 62.5 ms ] threadI use AWS and Azure. Each has their advantages and disadvantages. To answer if something seems "overly complicated" I would have to know, compared to what? AWS certainly has lots of complexity, but so does any services with so many features and moving parts and potentially catastrophic misconfigurations and security threats. Compared to managing servers in a rack I prefer AWS.
https://cloud.google.com/vertex-ai-workbench
That doesn't make any sense to me. What problems were you running into? Did you need to configure a bunch of super-non-standard stuff or something?
With people who use AWS SDK its all abstracted out. But there is time I just want to send a damn `curl` to download a S3 file and yes, doing the dance in bash isn't easy.
There is time I wrote a Lua plugin for openresty to fetch s3 and. I have to trial and error with lot of debugging. The ordering. the timestamp format...all of that...
The whole idea we had was stupid.
But if you’re saying clients will upload the wrong file, then that’s not really AWS’s fault is it?
If you’re saying the file itself is corrupted, I’m not sure how? The upload should fail if the content length is incorrect, and s3 doesn’t make the contents visible unless the upload completes.
You can use curl's `--aws-sigv4` option, then: https://curl.se/docs/manpage.html#--aws-sigv4
What you should be able to do:
Click click done. Upload files to S3. Point CNAME at AWS.
What you have to do:
Create an S3 bucket and stick the files in it. Create a zone in Route 53 and import your old zone file. Change your nameservers at the registrar. Wait a bit. Go to ACM in the correct region and create a cert. Tell it to add the DNS entries to R53. Wait a bit. Create a CloudFront distribution making sure that you get all the options right which is quite difficult. While that's deploying, copy the IAM policy it generates back to the S3 bucket. Jump back to R53 and add a new A record pointing at the cloudfront distribution alias. Cross fingers and start praying that the whole stack works. If it doesn't spend several hours working out which thing you forgot to click.
What devops culture would have you do:
Play with CloudFormation for 2 days writing oodles of mind numbing YAML and realise that you have to create the ACM cert and CF distribution in a different region to your bucket but that's impossible. Try and work around this with recommended StackSets but realise they are so horrible that they are unusable. Spend an hour googling. Download terraform because everyone is gushing over it. Spend several hours learning the above and writing it in HCL constantly tearing down and creating resources until the whole thing limps along. Eventually realise you need to share this is someone and the thing is stateful so pay Hashicorp for TF Cloud. After the PO is approved with takes 2 weeks, check it in to git, cheer loudly, hand it over to a colleague who has a different version of terraform and then discover about tfenv and terraform upgrades.
I only do this because I'm paid by the hour. I'm not even a cloud or ops guy. I'm an electronics engineer who needs to eat. I eat well due to this mess but I know it's all so so so wrong.
Doing that in Infra as Code shouldn't take two days IMO.
Every attempt keeps trying to solve the same problems with a new abstraction but the problem is the underlying abstraction not the tools.
It also involves writing many, many less lines of code than Cloudformation/Terraform thanks to the excellent abstractions it bundles in. Your example of writing "2 days' worth of YAML" is less than 120 lines of Typescript in CDK. You could even skip that and just npm install the AWS solutions construct for this scenario if you have literally no idea what you're doing: https://docs.aws.amazon.com/solutions/latest/constructs/aws-...
Edit: not strictly true. I know JavaScript quite well but I wish to have nothing whatsoever to do with it and the ecosystem around it due to a number of reasons I would have to write a whole essay on.
If you know CloudFormation, you do _not_ know the CDK. The CDK is split in 3 versions, ver 1, ver 2 and v2-alpha. Knowing the difference between L1, L2 missing options between the two.
Don't even get me started on docs. Docs in CloudFormation is KING. CDK's docs are a simplification at BEST. Most of the time I have to drop down into CF's docs to see what the actual constraints on a key are.
Given all my complaints, I'm not going back to CloudFormation. It is wayyyyy less code. Separated files. And, it's too hard to do complex stuff in CF. CDK has a lot of nice magic that takes away so much complexity esp dealing with cross region work.
Thus ends my corp-speak. The following is my opinion alone:
The thing that is hard for a lot of people to understand, it feels like, is that the CDK is both a library (nee runtime) and CI system in one. The runtime produces, at the end of the day, a CloudFormation template that gets shoved into the right places. The CDK happens to make generating those easier.
One of the core issues with working with CloudFormation is that it's limited in what the services support it doing. Services introduced before CloudFormation was really launched have the worst, but it wasn't until 2015 or so that most services really had "OK" support for CFN. CloudFormation is also brittle, and this causes a lot of headaches.
CloudFormation can't handle serial operations that require "create & permute" sequences -- where two things have to be set up in order, then glued together later after some kind of configuration. The way the CDK does this is by deploying lambdas that get invoked after your stack is shipped out that make the relevant API calls. Terraform does this by having the code that Terraform runs make the API call. The difference here gets to be that the CDK can synthesize a change long before it is actually effected, allowing for things like second order permutations. Terraform does everything in-situ, standing up infrastructure and taking the place of both the CDK and CloudFormation.
This has upsides and downsides on both, but a large upside (IMO, again) of the CDK approach is that you can code-wise validate certain things ahead of time before they're deployed, whereas with Terraform it's a little harder. Terraform's big upside is that it handles things that are Hard to do with CloudFormation alone by letting you figure out how to actually create/destroy that bit of infrastructure and hook up the relevant parts as one action in one codebase, rather than having to write Lambda handlers (the CDK calls these handlers "Custom Resource", which are different from Constructs) that get deployed and called when you need to setup/teardown/change the relevant parts.
Both Terraform and the CDK are opaque wrappers around AWS as a whole, the difference being that Terraform wrote their own DSL to describe things that gets then attached to their own SDK (which appears to be JS?) with varying levels of integration, whereas the CDK allows you to use whatever language you're already using to build the primary structure of your application, but where you have fully atomic (not just rollback-able) changes via CloudFormation.
[0]github.com/aws/aws-cdk/issues/new
Glad you included the "correct region" qualifier. The number of times I've mistakenly created the certificate in my local region, rather than the one designated for CloudFront...
Really these days I want to solve the problem and go home, not remember where the poo I stepped in last time I had to solve it was.
Ok I missed: point DNS to Hostgator name servers. Click the button to install lets encrypt.
The policies, cloudformation, r53 config, cloudfront all have their purpose and a lot of flexibility. But don't pay the complexity price if someone else can wrap it for you in a nice saas.
[1]: I'm probably going to move that to FreeBSD/OpenBSD with OpenBSD's relayd+httpd+acme-client at some point, but the general idea stays the same.
Deploying it? `aws cloudformation package -> aws cloudformation deploy`. That's it. No need for third party tooling or other madness.
I think the lack of understanding the fundamentals leads people to picking up tools that abstract the fundamentals away (but then leads to more complex solutions).
I am using a simple S3 bucket to serve image files.
We’re getting close to busting the free tier web traffic limit.
I can’t figure out where, or how, to enter my boss’s credit card so we can start paying for the service.
I’m not kidding. I’ve looked everywhere. Several times. I’ve probably spent two hours on this.
* We need a way to serve image files.
* I create an organization account, add an S3 bucket.
* We serve images as our application grows.
* Fast forward a few months, get an email saying, we're at 85% of free-tier limit.
* I invite my boss to the Organization, and make her the financial contact.
* Neither I, nor my boss, have been able to figure out where, or how, to enter a credit card number.
* We continue to get monthly 85% limit emails. Following the link in the email leads nowhere where we could enter the payment info.
* What a completely insane and stupidly opaque UI. Take our money! How?
* Will probably move to a service that's better organized. We're just serving image files, a few hundred per day.
We have both tried and are unable to figure out how to tie payment info to our AWS organization.
Did you give your boss a Billing role in IAM, within the Organization master account?
If so, search for the “Billing” service and Payment Methods will be on the side.
If in doubt, use the root account, but do so only sparingly. You can use it to debug admin and billing permissions.
https://github.com/lifebeyondfife/simple-static-website
I created account after account at first and then had hell deleting them entirely. was still easier than figuring out was using a vpc that had nothing visible attached to it
That's the point - I do this too. I like the fact that AWS jobs pay well. I'm fair and I always point out to my clients that AWS is expensive and it will cost them more in the long run, and the vendor lock-in is considerable - but they don't care as "everybody is doing it." Well, that's fine with me, I also like to eat well.
> Play with CloudFormation for 2 days writing oodles of mind numbing YAML and realise that you have to create the ACM cert and CF distribution in a different region to your bucket but that's impossible.
The discrepancies in cross-region support specifically with ACM drives me nuts.
I noped away from Terraform back in 2019. At the time, the state management component was too immature, and it was a little.. lose with credential security.
You may like Ansible far more if you’d like something a bit more declarative, readable, and with “batteries included.” I have a close friend who is an EE and he finds tools like Ansible much more intuitive and enjoyable than Terraform and CloudFormation.
Every tool has its master.
Most people don't build their signatures by themselves - they use the SDK provided.
AWS documentation isn't consistently good, and there are lots of decoys. So, if you wound up in some local minima that made it look like you had to implement the signing and such yourself, try a Web search that includes the name of your chosen programming language.
(A long time ago, I had to implement the AWS client API from scratch, but that's because I was using a fringe language that didn't have such libraries.)
https://curl.se/docs/manpage.html#--aws-sigv4
Actually, security is not supposed to be complicated. "Complicated" is the anti-thesis of "secure".
It did not tree shake cleanly with my build system and I eventually ended up just yanking AWS from the stack entirely.
So... if it is out there, it's complicated enough I couldn't find my miss-steps then or now.
https://www.npmjs.com/package/@aws-sdk/client-secrets-manage... for example
Another example is Google cloud run vs elastic container service. Cloud run is dead easy to get up and running. ECS is a mess of confusing terminology and unnecessary complexity. I literally do not care what class of machine you will use to run my container. If I cared that much I would have used a proper EC2 instance.
However, having said that, compared to FAANG internal tools, its actually not that bad. It's at least vaguely consistent
It's because it is an "internal tool". All services are built on top of aws native.
An internal service is really good and then it becomes an AWS service.
I've even been in 2 teams that have tried to do that and one may succeed in a few years.
https://github.com/canercetin-randomguy/club-noira/blob/main...
This is only the auth&sign part. and it doesnt work. and it is already 150 LoC. god.
Actually I find a lot of Amazon's internal tools to be a breeze to use compared to AWS.
Basically all the painful Amazon-specific stuff is done for you so it makes everything very easy.
Some stuff is bad obviously but what we use the most is pretty freaking good.
Also: All of this is handled in the SDKs. Anyone implementing this themselves either isn't using the right libraries or has a very special use case.
Anyway, this is from the perspective of a large service provider. For side projects or smaller services, yeah simple can be good. But it really depends on your security requirements, threat model, customer requirements, etc.
Authentication doesn't require signatures, that's my point. They often place the burden of getting the security right on the client.
while bearer tokens are simpler, the protection they provide is more limited. for example, a bearer token has no request affinity so if i get your token for service foo from a request you made, i can impersonate you for the lifetime of that token. and it’s much easier to get your token, because it has to live your physical machine to be of any use.
with bearer tokens, the only way to reduce blast radius is to generate more tokens. get a token per-service/region, get a token every 5 minutes, etc. this is possible, but comes at the expense of availability. if the token vendor is down, the impact of the outage is proportional to how frequently and in what situations a caller needs to get a new token.
with signatures and client computation of scoped keys, you’re able to use much longer lived credentials while generating short term credentials only where necessary for that request.
And if you’re building your own SDK (why?), well there is a lot more complexity down the line once you get past authentication.
There are more languages than C++, Go, Java, JavaScript, Kotlin, .NET, Node.js, PHP, Python, Ruby, Rust & Swift.
But if you’re building a feature to interact with AWS then ignoring the availability of SDKs is stupid. If you want to write your service in brainfuck then go for it, but don’t blame AWS for that decision.
Also, AWS not having something like Azure's resource groups or GCP's projects and instead telling you to create accounts for different projects and environments, and using AWS Organizations or however they call it sounds like a huge PITA.