Had a similar issue with CloudTrail logs. These are delivered to your S3 bucket but the objects are owned by AWS's 'CloudTrail account'.
This means you can't drop the logs into your security account and then query them with Athena from another account.
Took ages to figure that out.
Not a lot of issues really. Lots of lessons learned and small features I wish existed.
Here's a classic: I had a lambda fed by an even stream hanging off a Dynamo table. The lambda would delete the record after doing a bit of stuff. I didn't realize on my first pass that this would trigger the lambda again... Wasn't a big deal as I noticed the behavior pretty quickly.
An issue I've encountered with static website hosting on CloudFront+S3 is that if you want `index.html` to be served at the root path of a URL - e.g. https://www.yoursite.com/folder/ vs https://www.yoursite.com/folder/index.html, you must use the s3 website endpoint rather than the S3 Origin. And in that case, unfortunately, it is not possible to use Origin Access Identity.
"It does not work on any subdirectory (such as http://www.example.com/about/). If you were to attempt to request this URL through CloudFront, CloudFront would do a S3 GetObject API call against a key that does not exist."
I just learned this same thing (Object ACLs) last night, and my mind was blown that a bucket can have objects owned and controlled by other accounts, which then prevents a third account from accessing those objects even though it has bucket access.
But when you stop and think about it, it's a similar model as Unix file/directory ACLs...I just hadn't expected it to apply to S3 buckets too.
I still have the tab open for the docs to fix access:
For anyone else that is looking at rolling your own static hosting like this on AWS specifically (including CloudFront cache invalidations and protected domains), I highly recommend checking out Amplify Console https://aws.amazon.com/amplify/console/. It's a little bit more feature-rich and avoids headaches like the article ran into. Closer to Netlify than it is to CloudFront + S3.
Does Amplify Console support the multi-account usage pattern described here? That's the part responsible for most of the complexity + headaches in this setup.
Yep, and it does so pretty easily. You just connect Amplify Console projects in each of the two accounts to a particular branch (or branches if you want feature branches in dev) of your git repository.
multiple account support in amplify involves configuring amplify in each account to a particular branch. amplify doesn't support CI/CD types of deployment - instead, you would need to push to a 'dev' branch to triggers a dev build and then later push to a 'prod' branch to trigger a prod build. one reason we went with a central pipeline is to have a single account to manage all deployments as well as control what code gets to production.
Honestly after using Amplify to develop a new app, I have a hard time recommending it unless your use-case fits their design perfectly and will never grow beyond those bounds.
If you've already made any design decisions or need to do something differently than Amplify expects, it is a minor nightmare. It generates multiple local copies of templates for nested cloudformation stacks and doesn't really support manual editing. The basic Amplify CLI commands entirely abstract away what's actually happening with the AWS APIs. "Ejecting" takes loads of time to basically reverse-engineer the Amplify CLI, clean up the generated templates, and then create your own build process that you can tweak.
It would be much easier to recommend Amplify if it wasn't so far removed from the underlying AWS APIs and it provided an easy way to modify those API calls or templates instead of being stuck within its guard rails.
Classic AWS. I have a SA Pro cert and I still couldn't reliably articulate the difference between Lightsail, SAM, and Amplify CLI. It feels like they keep adding to each without any apparent strategy. I didn't even know there was another product also branded "Amplify" separate from the CLI.
AWS really needs more centralized decision-making. It feels like individual product teams are allowed to go way off in the weeds without any frame of reference for whether their solutions make sense when plugged into the full suite.
The Amplify naming is a special kind of AWS classicality. Amplify, what a great Netlify like experience! Oh... not that one, you mean the CLI/Framework. Also pretty cool, we'll see how it plays out with the CDK. No? You mean the literal Amplify library that makes UI work easier. All of which can work together, but don't necessarily have to.
I can't complain too much, it's a bit of job assurance for folks like myself that specialize in only AWS. Exhausting keeping up to speed on everything though.
+1 on this. wanted very much to like amplify but after using it for a few small projects, decided it was not worth the trouble. it abstracted too much and was too opinioated for what i wanted to do.
the one use case i do like it for is hosting simple static sites (eg. jekyll based sites).
Cool write up. Just went through this exact use case a couple months ago. I wish there was a simpler way of just making the bucket owner automatically own anything put into that bucket to simplify use cases like this.
25 comments
[ 0.36 ms ] story [ 63.1 ms ] threadIt feels like a common pattern to consolidate your logs into a single account for analysis, I wish they made it more straightforward to setup.
One workaround I've heard of is S3 replication. I've not tried it yet but apparently the replica has the 'correct' ownership.
Here's a classic: I had a lambda fed by an even stream hanging off a Dynamo table. The lambda would delete the record after doing a bit of stuff. I didn't realize on my first pass that this would trigger the lambda again... Wasn't a big deal as I noticed the behavior pretty quickly.
"It does not work on any subdirectory (such as http://www.example.com/about/). If you were to attempt to request this URL through CloudFront, CloudFront would do a S3 GetObject API call against a key that does not exist."
But when you stop and think about it, it's a similar model as Unix file/directory ACLs...I just hadn't expected it to apply to S3 buckets too.
I still have the tab open for the docs to fix access:
https://docs.aws.amazon.com/AmazonS3/latest/dev/example-walk...
> > When you see hoof prints, think horses not zebras.
Looked this one up and the original medical aphorism dates back to the 1940s and was 'When you hear hoofbeats' [1], in case anyone's curious.
[1] https://en.wikipedia.org/wiki/Zebra_(medicine)
https://youtu.be/fE2KDzZaxvE
Zebras All the Way Down - Bryan Cantrill, Uptime 2017
If you've already made any design decisions or need to do something differently than Amplify expects, it is a minor nightmare. It generates multiple local copies of templates for nested cloudformation stacks and doesn't really support manual editing. The basic Amplify CLI commands entirely abstract away what's actually happening with the AWS APIs. "Ejecting" takes loads of time to basically reverse-engineer the Amplify CLI, clean up the generated templates, and then create your own build process that you can tweak.
It would be much easier to recommend Amplify if it wasn't so far removed from the underlying AWS APIs and it provided an easy way to modify those API calls or templates instead of being stuck within its guard rails.
AWS really needs more centralized decision-making. It feels like individual product teams are allowed to go way off in the weeds without any frame of reference for whether their solutions make sense when plugged into the full suite.
I can't complain too much, it's a bit of job assurance for folks like myself that specialize in only AWS. Exhausting keeping up to speed on everything though.
the one use case i do like it for is hosting simple static sites (eg. jekyll based sites).
that being said, i do like the amplify javascript sdk, especially the auth library and react components: https://aws-amplify.github.io/docs/js/api
best part is that you can use the components without using the amplify service/console