Launch HN: Slauth (YC S22) – auto-generate secure IAM policies for AWS and GCP
Check out the video or give our open-source CLI a try with one of the sample repo's on https://github.com/slauth-io/slauth-cli https://www.loom.com/share/bd02211659eb4c7f9b335e34094b57cb?...
We got into the cloud access market by coincidence and were amazed by the amount of money spent on IAM. Current tooling such as http://Ermetic.com and http://wiz.io/ visualize IAM misconfigurations post deployment but don't actually change engineering behavior, leaving organizations in a constant loop of engineers deploying over-permissive policies ⇒ security engineers/CISO's getting alerts ⇒ Jira tickets created begging developers to remediate ⇒ New over-permissive policies being deployed again.
We interviewed hundreds of developers and DevOps engineers and discovered two key pain points:
1. *IAM is a Hassle:* Developers despise dealing with IAM intricacies. 2. *Speed vs. Security:* IAM was slowing them down in deploying quality code swiftly.
So the objective is automate policy creation so that developers don't have to deal with it, and harden IAM security pre-deployment.
We employ Large Language Models (currently OpenAI GPT-4) to scan code in any language. Through a series of prompts, we identify service calls and the actions required. The resource name receives a placeholder unless its embedded in the code. We aim in the future to create a static code analyzer in order to not send any source code to LLM's but for now using LLM's is the fastest way to market and somewhat accepted by the industry through the use of Github Copilot etc.
You can use the CLI in the terminal or actually integrate it in your CI/CD and have it become a part of your development team workflow.
Three main questions we receive
1. *Security Concerns:* How can I trust [Slauth.io](http://slauth.io/) to access my source code? 2. *Policy Accuracy:* How can I trust [Slauth.io](http://slauth.io/) creates the right policies? 3. *Differentiation:* How are you different from IAMLive, IAMBic AccessAnalyzer or Policy Sentry?
To address the first concern, we don't access your code directly. Instead, we offer a CLI that integrates into your CI/CD pipeline, allowing local code scanning. http://slauth.io/ uses your OpenAI key to convert the code into a secure policy, with the option to output results to *`stdout`* or a file for artifact upload and download. That does mean OpenAI has access to the source code located in the path you set to be scanned as we need to know which SDK calls are performed to generate the policies.
We have extensively tested it on AWS , Typescript and GPT 4 with very good results (>95% accuracy). We do know these accuracies drop when using GPT 3.5 so if possible, use GPT 4 as we are improving the prompts. GCP and Azure have been tested less but the results when using GPT 4 seem equally high. We also have experienced some hallucinations but they have not effected the outcome of a secure policy but merely the structure of how the policy is generated. That is not to say that it is 100% reliable henc...
78 comments
[ 4.8 ms ] story [ 138 ms ] thread> We employ Large Language Models (currently OpenAI GPT-4)
For IAM, this seems like a disaster waiting to happen. Combining hallucination problems with security settings is not a path I would consider
My concern is for those who blindly trust LLMs. Security posturing is not the place to be an early adopter of AI tools. You have to understand both IAM and system architecture to know if what the LLM is saying is correct, so where does that leave us?
I think they can be an extra pair of eyes, but not the driver. Still, there is a signal to noise problem that remains, due to the inherent hallucinations.
my policies are definitely too broad, but feels like I should be able to tighten them up without changing code. (just potentially breaking things if I get it wrong and go too tight).
1. The application has to start using credentials for the first time, or consume them a different way. For example, stop consuming an environment variable and rely on a service account.
2. You have to change ops to support new workflows. Often you have to put approval workflows in place because fewer people can do things and you want only the machines touching production
3. You have to change human behaviors and habits (this is the real hard one). I've had to revert changes because the increased security blocked developers and they don't have time to adapt for the next deadline.
4. Getting parity in local development workflows is also challenging. How and where do you match vs except from IAM parity?
5. Should I give the current server access to a particular cloud service/resource or break out that particular function into a lambda and minimize the permissions there? You have to think through the implications of a breach and how/where you want to limit the blast radius.
6. This is probably obvious, but implementing application level controls, like API endpoint permissioning. IAM is not limited to cloud infra
ask questions to deepen your understanding
> ...ping-pong...
It was a scheduling problem rather than a decision problem. The impact radius is always more than you anticipate
I'm not saying don't use them, just use them as an extra pair of eyes, mostly to catch errors rather than to drive and architect
> get it that there is a lot of skepticism because of people hyping LLM's
The skepticism is not from the hype, it's from experiencing LLM output personally. They are fine if the output can be fuzzy, like a blog post or a function signature, not so much if there is a specific and fragile target.
I can justify to management that we have limited time for IAM and something was missed that we can fix / create tests / scans for after an incident. It's harder to explain that we chose a vendor that uses a non deterministic tool that can hallucinate for one of the most core security pieces of the puzzle.
Similarly, LLMs used for SQL generation meant for business analytics is also a critical area where if numbers are wrong, it might lead to a business going bankrupt.
For Prototype, fun exercise, sure go all in.
It'd potentially cost millions more to recover from a GPT-4 disaster.
These would be the "s3:*" and "Resources: *" scoped permissions I assume? I can't imagine users are explicitly typing out permissions, 95% of which are not relevant for the task.
> which creates huge problems
Such as? What is the material impact of a workflow or a user having too many permissions?
> and is a reason for spending millions in security tools
Are you claiming that overscoped IAM permissions alone are responsible for 1M+ security tooling bills in companies? Would you be willing to share information on which tools these are?
Security obviously https://en.wikipedia.org/wiki/Principle_of_least_privilege
How many times have excess permissions "actually" been the problem... versus something like correct permissions with compromised credentials?
By the way, I have no affiliation with slauth.io (just found them today as well). I just think that https://en.wikipedia.org/wiki/Principle_of_least_privilege is something good to follow in critical systems.
A few examples I've seen repeatedly:
* An AWS-hosted container/artifact/CI/CD application has an SSRF vulnerability that can be used to retrieve IAM instance credentials. Because micromanaging permissions is hard, and the application needs to access so much content in S3, spin up/down instances, etc. it has ec2:* and s3:. Unless the organization has created a separate AWS account for this platform specifically, it's probably game over at that point.
An internet-facing MDM solution has a code execution vulnerability. Because the vendor didn't want to document all of the individual permissions it needs, the installation instructions specify that it should run as an account with Domain Admin permissions in AD. That is definitely game over for most organizations, because even systems that don't authenticate against AD are almost always accessed from systems that do.
Micromanaging permissions is hard in a big organization. I saw it done well, years ago, in Active Directory, but it took several FTEs who were personally interested in the topic to set up and manage, and that was a traditional big business IT environment. In a startup-style free-for-all, good luck. I don't have an opinion either way on Slauth specifically, but something that generates IAM policies procedurally seems like a step in the right direction.
If you want to hear about stories of privilege escalation there should be easy to find. I also have some on my own which I might describe in another post but essentially it was the classic - CI/CD pipeline that "thinks" it has access only to QA does a "destroy all servers" in both QA and Production because it also had access to production without knowing anything about it.
"Accidentally destroyed production database on first day of a job"
At my job for example we don't have a separate support team for the ETL work we do so I have a lot of access I don't use unless things are breaking and then I can't wait for the access approval process to get added to database XXX or bucket YYY to diagnose what data has broken our processes.
I wrote some code once to fetch all those preconfigured role permissions and then present them in a more digestible way
Anecdotally I've used copilot to help write a lot of IAM polities in Terraform and the accuracy is basically 100% already.
With an API that has a hidden / predefined prompt, you'll run into hallucinations that are harder or impossible to handle
My general approach is to spend more time up-front, so when you are in the heat of a problem, you don't have to learn under pressure. I think my beard is graying
Regardless, anything to make IAM provisioning easier is worth a go as long as you verify the results using a simulator or something.
Again, I understand the skepticism using LLM's but currently everything is done manually and it shows that doesn't work well. So using LLM's is a quick way to improve the current situation and hopefully we can further compliment it with checks and balances
If it is all done manually, and there are both good and bad IAM setups, can you really extrapolate to "manual" being the root cause? How can you even get an LLM to produce secure policies without having existing secure policies to train on? The entire premiss seems off and misleading to me
I would expect a hands-off approach to have worse outcomes
The big disconnect here is I can't share code with OpenAI for various reasons.
Would you consider using something like AWS Bedrock+Anthropic Claude, where we have better (more predictable/risk profile) control over data-sharing, etc?
But I'm sure others are ok with it, so that's great.
A challenge with Slauth will be to organize the generated policies in a way that makes them legible. I would like the IAM policy to help clarify its intent. Allowing each in-use API endpoint is technically required to let the service work. It might be technically following the principal of least privilege. But the endpoint-by-endpoint rules do a poor job of summarizing the purpose of the policy or how it relates to the service. One way that we do this is by having resource providers declare managed policies that allow use of the resource. So the "encabulator" provider also defines a "mutate encabulator" managed policy. Then services that need to invoke "mutate encabulator" can reference the managed policy. They don't need to compute the ARN of every endpoint. The dependent service doesn't end up with an inline policy that has detailed dependencies on the implementation details of each target resource.
On another note, are there tools that will scan your AWS/GCP logs and emit configuration to limit permissions to what you're actually using? I could even see GPT doing better here too, or at least it would be easier to test.
1. The service needs to be deployed 2. To produce an actual result, the calls that make use of the sdk need to be triggered
This is something that would be better included as an addition to monitor policy usage and adjust.
And probably some quality-of-life wrappers around all that process?
Just as an example, I setup a containerized app on Fargate with a custom role, and the need to configure the ability for ecs to assume the role, read from ecr, write to cloudwatch, create channels for debugging was super annoying.
Comparatively, having a policy for it to read from an s3 bucket with my data was trivial.
Just sharing my story. IAM sucks and GPT-4 is a good backup for configuring it.
I do agree that everything about it is horrific, though I'd be surprised and impressed if an LLM were able to generate your key setup from scratch.
1. The tooling you're quoting specifically, wiz.io and ermetic.com do an incredible amount more than just "scan for IAM misconfigurations". In fact, I understand that to be one of their most insignificant features. Yet it sounds, from the numbers being quoted (I saw the "millions" figure being thrown around), that you are equating a company purchasing wiz.io as them purchasing "tooling to scan for IAM misconfigurations" exclusively. How much does the IAM scanning tooling actually cost, and what is the material cost of delayed remediation of over-permissioned entities?
2. Were a company to introduce Slauth into their stack, are you under the impression that they would then not need to scan their IAM for misconfigurations and would therefore be able to save "millions"? Would it not be fair to say that the presence of Slauth would not remove the need for IAM scanning tools, since IAM deployments could happen out of bounds, which is not something that Slauth removes from a companies threat model?
The quote you use got me to further research the market and speak to users of those toolings. From speaking to the users it was evident that the amount of misconfigurations being deployed wasn't being reduced.
I imagine users of cloud scanning tools would also use a pro-active tool like Slauth or any other shift-left tool that would aim at preventing as opposed to reacting.
At the same time, it will create noise in a PR, it will be reliably wrong, so it is not really about saving time, but more about always having an extra (junior) reviewer, it's only going to catch the simple things. You will have to work hard to improve the signal to noise ratio. Your current examples are all very simple and do not reflect real world complexities. I'm very much doubt ChatGPT has enough context length for devising real world IaC from source code, or even checking it. How many times will the code changed in a PR require knowledge about code not changed in a PR?
I'm curious though how well an LLM performs for newly released AWS services? This is where I've experienced the most arcane IAM definitions personally, but I wonder if GPT 4 is trained well enough on newer sources.
Does this support resource policies? If so, how are you ensuring serious security vulnerabilities such as confused deputy are not introduced by this tool?
As is stands, Slauth doesn't support resource-based policies.
I'd also be curious for future plans with resource policies as that's another layer of complexity to manage - where the resource policy would manage access to potentially many applications -> 1 resource. Vs 1 application -> many resources which I think is the use case Slauth is solving for initially.
Confused Deputy would be interesting, could be done via Condition Keys such as SourceArn and SourceAccount, but gets complex for cross-account use cases.
Another use case you might run in to as you talk with more clients is figuring out what developer IAM roles need to be. This was the far bigger problem for us as we had a ToS that restricted employees from viewing/accessing user data.
There's a couple different models for IAM ownership. At some places, the application teams own IAM along with the application. Sometimes, it's owned by central teams (such as security).
And agreed, with companies growing and changing, ownership changes as well.
Those factors can all complicated IAM development and policy maintenance as it becomes more difficult to find the right fit for IAM to application. For that, it would require someone who knows exactly what the application needs access to and the IAM actions taken as well as how to configure IAM.
Also at cloud-scale 95% accuracy is completely unacceptable.
Datadog recently did a State of Cloud Security and one of their findings in https://www.datadoghq.com/state-of-cloud-security/ is that a substantial portion of cloud workloads are excessively privileged (with more data points there).
Oh you sweet summer child. But in reality I've seen the pattern over and over, especially in GCP. 1. Create service account 2. Give it Owner permission for the whole project 3. Download key and check it into source control 4. deploy.
Instead of analyzing your code, you just run your code with no permissions and it automatically detects permission failures and thens open those permissions, with a UI showing you what it did so you can remove any permissions you don't want.
That actually seems much more secure than trying to divine the rules from reading the code.
What value is the LLM adding here?
I'd like to think most investors are sophisticated enough to detect when the "AI" was just bolted on for funding, and that most startups aren't actually doing that, but are using LLM for a reason.