[Edit] too funny, the first three comments are all similar messages with three different communication styles (this one the most terse, I have much to learn)...
> The AWS account takeover was possible because a highly privileged IAM role had been assigned to the EC2 instance running the vulnerable Docker container app. While the Log4j2 vulnerability allowed initial access to the Docker container, the privileged IAM role enabled lateral movement and ultimately a total compromise of the AWS account.
Saving you a click... who would realistically give such a high permission set IAM to an EC2?
Where would the giant red alerts stop? They already have it for S3 after so many people left their buckets open. But is it also not our responsability to properly understand the tools we use?
They should stop where there is no risk. Consider commercial aircraft. They warn pilots of risks because despite having thousands of hours experience, humans (pilots) make mistakes.
> But is it also not our responsability to properly understand the tools we use?
For that, those who built the tools must communicate how the tools work. It's not our responsibility, nor is it even possible, to read the mind of the people who made the tools.
AWS is so difficult to use securely by default that there is an entire thriving industry of vendors who will analyze your permissions and try to bring them down to meet the standard of principal of least privilege. I'm not at all surprised that there are many insecure environments and privilege escalation exploits; AWS is almost aggressively hostile to any attempts to use it securely.
This is completely untrue. AWS offers many tools to audit credentials. It also has the policy simulator to see exactly what the impact of changes will be on a role.
> This is completely untrue. AWS offers many tools to audit credentials. It also has the policy simulator to see exactly what the impact of changes will be on a role.
Your comment fails to prove, or suggest, that OP's claim that AWS is too difficult to use securely by default is false. To first be in a position to refute OP's point you would need to prove, for starters, that no such vendor exists, which is absurd because they do exist, don't they?
In fact, if anything it supports OP's claim, as you've just pointed out that AWS even tries to profit from their problem of making it too difficult to use it securely by default by selling a premium service to audit credentials.
Minimizing AWS permissions to just the set you need can be a royal pain in the ass. Many of the canned AWS-managed example policies end up giving either ANY action on a resource, or grant a handful of actions on ANY resource, (e.g. grant S3:* on some bucket, which inadvertently lets you delete the bucket too)
Simply reading the docs and attempting to divine the right subset of permissions can be nearly impossible. Usually you must guess, then exercise the software until it fails, then look in CloudTrail to see which permissions got denied, then try a new covering set, and repeat until nothing breaks. To say this is frustrating would be an understatement.
This is a helpful writeup in terms of specific queries to look out for, but any RCE on a host with privileged IAM access is going to lead to this scenario.
> The AWS account takeover was possible because a highly privileged IAM role had been assigned to the EC2 instance running the vulnerable Docker container app
This attack isn't unique to Log4Shell; it's a symptom of giving your (compromised) EC2 instance global admin access.
To me this is less about log4j, and more about giving everything root.
> The AWS account takeover was possible because a highly privileged IAM role had been assigned to the EC2 instance running the vulnerable Docker container app
The mistakes are, in order:
1. Binding an administrator IAM role to an EC2 instance, which is never ever a good thing to do, and
2. Running a docker container with full root privs - docker is not as much a security barrier as you think it is - it's only slightly better than running the application as root on the VM itself.
So yes, the log4j vulnerability is dangerous, but not nearly as dangerous as running everything as root all the time.
I just grabbed by copy of CGI Programming on the World Wide Web by Gundavaram from 1996, and on page 368 it says:
> Most servers are set up to run with the user identification (UID) of "nobody," which means that your scripts have to be world executable. The reason for this is that "nobody" has minimal privileges.
Sure, there would always have been a few idiots who ran everything as root, but my recollection, backed up by the well-respected O'Reilly & Associates here, is that running internet-facing services with restricted privileges was the majority position for at least as long as web servers have had version numbers of 1.0+.
But the book isn't saying "You should try to make sure your web server runs as nobody". It's saying "when you're deploying your CGI programs, take into account the fact that the web server you're deploying to likely runs as nobody".
It takes real effort to run most of the LAMP stack as root. If you don't understand Unix permissions it's not easy. Apache, mysql, (and therefore php) all run by default as a low-privileged user. I have plenty of real world experience and haven't seen apache or mysql run as root outside of embedded devices, and even that was maybe a handful of times.
I think this might be a newer problem than 1996 actually. I have seen an awful lot of "modern" (2010s at least) devops which runs everything as root because the developer could not be bothered to understand the unix permissions model. I would guess that focus on security is something that waxes and wanes, rather than something that monotonically becomes more important in the culture.
It should because if you can exploit the root user, it is much easier to use that machine as a lunching point for a secondary attack. Further, root will let you cover your tracks much more easily than an unprivileged user.
Both are problems, but to me the biggest facepalm goes to log4j because anyone in their right mind should assume that log messages are untrusted input and should absolutely never have been parsed for control sequences. It's another prime example of Java over-engineering everything that should have been simple.
Don't blame the language, blame the library. At least it's not part of the standard library - in C, if you accidentally pass a user-provided string to the first argument of printf() you could be in just as much trouble.
There's a lot of over-engineering and tolerance of complexity in the Java world, more than in many other ecosystems, it's a pretty widely held view. The library is a product of the environment it evolved in.
Log4j is not java. In fact, you can have log4c# or log4rust or log4fortran with precisely the same design, and consequently the same design problems and vulnerabilities.
So if they didn't create a new user account and IAM account what would you see? If they just used the remote shell and the installed aws cli e.g. `aws s3 ls` would you be able to detect it?
This article is an ad.
You'd still see the activity of that machine in AWS CloudTrail logs.
From [1]: "CloudTrail records two types of events: Management events capturing control plane actions on resources such as creating or deleting Amazon Simple Storage Service (Amazon S3) buckets, and data events capturing data plane actions within a resource, such as reading or writing an Amazon S3 object.
36 comments
[ 4.9 ms ] story [ 41.2 ms ] thread> The AWS account takeover was possible because a highly privileged IAM role had been assigned to the EC2 instance running the vulnerable Docker container app. While the Log4j2 vulnerability allowed initial access to the Docker container, the privileged IAM role enabled lateral movement and ultimately a total compromise of the AWS account.
Saving you a click... who would realistically give such a high permission set IAM to an EC2?
Perhaps AWS should create a giant red alert that customers must acknowledge before applying such a configuration.
For that, those who built the tools must communicate how the tools work. It's not our responsibility, nor is it even possible, to read the mind of the people who made the tools.
Your comment fails to prove, or suggest, that OP's claim that AWS is too difficult to use securely by default is false. To first be in a position to refute OP's point you would need to prove, for starters, that no such vendor exists, which is absurd because they do exist, don't they?
In fact, if anything it supports OP's claim, as you've just pointed out that AWS even tries to profit from their problem of making it too difficult to use it securely by default by selling a premium service to audit credentials.
Simply reading the docs and attempting to divine the right subset of permissions can be nearly impossible. Usually you must guess, then exercise the software until it fails, then look in CloudTrail to see which permissions got denied, then try a new covering set, and repeat until nothing breaks. To say this is frustrating would be an understatement.
The same people who think docker/k8s is what everyone should use and that java is a slow language?
> The AWS account takeover was possible because a highly privileged IAM role had been assigned to the EC2 instance running the vulnerable Docker container app
This attack isn't unique to Log4Shell; it's a symptom of giving your (compromised) EC2 instance global admin access.
> The AWS account takeover was possible because a highly privileged IAM role had been assigned to the EC2 instance running the vulnerable Docker container app
The mistakes are, in order:
1. Binding an administrator IAM role to an EC2 instance, which is never ever a good thing to do, and
2. Running a docker container with full root privs - docker is not as much a security barrier as you think it is - it's only slightly better than running the application as root on the VM itself.
So yes, the log4j vulnerability is dangerous, but not nearly as dangerous as running everything as root all the time.
When was that, exactly?
I just grabbed by copy of CGI Programming on the World Wide Web by Gundavaram from 1996, and on page 368 it says:
> Most servers are set up to run with the user identification (UID) of "nobody," which means that your scripts have to be world executable. The reason for this is that "nobody" has minimal privileges.
Sure, there would always have been a few idiots who ran everything as root, but my recollection, backed up by the well-respected O'Reilly & Associates here, is that running internet-facing services with restricted privileges was the majority position for at least as long as web servers have had version numbers of 1.0+.
Because in an alternative universe, they could have chosen to treat log messages like SQL where parameters are passed separately.
Obviously some people must believe that logs should be trusted input, otherwise we wouldn’t be in this situation.
That said I consider both logs and error exceptions as untrusted input, but purely on practicality.
They did chose to do this. The vulnerability arises even when this is done correctly.
Pseudocode:
This is still vulnerable. The parsing that exposes the vulnerability occurs on both the format string and userInput here.Log4j is not java. In fact, you can have log4c# or log4rust or log4fortran with precisely the same design, and consequently the same design problems and vulnerabilities.
From [1]: "CloudTrail records two types of events: Management events capturing control plane actions on resources such as creating or deleting Amazon Simple Storage Service (Amazon S3) buckets, and data events capturing data plane actions within a resource, such as reading or writing an Amazon S3 object.
[1] https://aws.amazon.com/cloudtrail/features/