One of the things we do at Sym is help customers replace risky permanent access with just-in-time, approval-based access. Sometimes this is part of a customer’s broader initiative to scope down policies across the board to achieve least privilege access. This process can be as painful as it is important, which is probably why there are tools popping up that make defining least privilege policies easier.
This article explores a few tools that use AWS logs to help you create least privilege policies. Getting to the right policies can be an ongoing game of whack-a-mole and no tool can generate perfect policies with no effort. They all require you to put thought into your approach and do last-mile tweaking but they can help with the toil.
Using logs to inform policies generally involves a retroactive analysis but at the end of this article I'll explore a way to determine Actions required for running Terraform without the need to run terraform against your AWS account. But first let's dive into some cool data-driven policy wrangling tools starting with a few that use CloudTrail.
By collecting event names, sources and resources from CloudTrail events, you can identify permissions that enable only those specific events and use them to create a least privilege policy.
Alex Smolen has a great post demonstrating how you can analyze CloudTrail logs into Athena. This is useful for getting a sense of access patterns to inform your policies, but if you’re able to target a specific IAM entity and a time range, you can use a tool like TrailScraper to automatically construct a policy directly from those events.
There are a couple of gotchas with using CloudTrail events to build policies. First, the event names don’t always line up exactly with the IAM Actions. You can mitigate this by using a tool like TailScraper, which accounts for misfit event names. The second gotcha is that CloudTrail doesn’t log data events by default which is probably a good thing for your AWS bill, but might result in some missed Actions in your policy.
If you’ve spent much time managing access to cloud infrastructure, you know that getting your policies in place is only the beginning. Services change, teams change and code changes. Pretty soon someone will need a permission not found in their policy and will receive an "access denied" or "unauthorized" error. Access Undenied on AWS is a handy tool to help you keep your policies up to date for your team. It will look for “access denied” events in CloudTrail and provide missing permissions to help you keep your team moving.
If you want to get serious about crafting policies from CloudTrail data, you may find yourself looking into IAM Access Analyzer. IAM Access Analyzer was launched in 2019 to help spot unintended public and cross account access but last year Amazon added the ability to generate policies from CloudTrail logs. IAM Access Analyzer policy generation is free and full featured. It does have its own gotchas but they are well covered in the “Things to Know” section of the docs.
It's worth noting that once you've created your policies, there are many ways to keep them scoped down. AWS Access Advisor helps you refine permissions by using last accessed information to identify and remove unused permissions. AirIAM uses Access Advisor to automate this process and get those polices into Terraform. Netflix's Repokid also uses Access Advisor (via Aardvark) to automatically remove unused permissions.
Another source of event logs useful for informing access policies is Client Side Monitoring. It’s what powers AWS CloudWatch, but if you enable CSM in your AWS client you can capture the data for yourself and generate policies similar to how you would with CloudTrail logs. cloudonaut has a great post on how to enable AWS CSM on EC2 and analyze the logs using Athena.
But what if you want to figure out what permissions you need to execute some Terraform? Well, because the AWS client powering the provider also supports CSM, you can enable CSM when running terraform apply. If you run AWS_CSM_ENABLED=true terraform apply, you’ll get logs sent to the loopback interface on port 31000 via UDP which will include information from which permissions can be derived. A fantastic tool to help go from raw logs to policies is iamlive. Just kick it off in another terminal, run your Terraform and watch the policy materialize.
.gif)
OK, so which one of these two data sources is better for deriving least privilege policies? It of course depends on your specific needs, but there are a couple advantages to CSM. First, it collects API calls that CloudTrail can leave out. Second, while CloudTrail enables retroactive analysis of required permissions, CSM provides the potential of leveraging LocalStack so you don’t need to run Terraform against your actual AWS account to generate policies. Let’s run through a demo of how that could work.
Let’s fire up a terminal and get iamlive going:
In another terminal, we can pull down an example terraform file that I created and use LocalStack to mock out AWS. We want to generate logs that will be picked up by iamlive.
Now you have a policy in your iamlive terminal!
A downside to this approach now becomes apparent. If we want to scope down the Resource we’d need to do it manually and using LocalStack makes that more difficult. I included an output block in main.tf to capture the ARN of the EC2 instance, but a LocalStack generated ARN is not really useful when crafting a policy for your AWS Account.
However, if it would be useful to quickly determine the relevant Actions for a given terraform project, using LocalStack with CSM has a significant benefit. Subsequent terraform applys and terraform destroys will generate different Actions than the initial terraform apply so you might want to run through different scenarios to capture the right Actions. LocalStack allows you to do this without hammering your AWS Accounts.
Check your iamlive terminal and you'll see that your policy now includes Actions required for terraform destroy.
Note: don’t forget to shutdown LocalStack when you’re done.
This approach has worked well for me with a limited set of API calls I’ve used but your own results may vary. LocalStack has done an amazing job of mocking out AWS APIs but depending on the AWS services you’re terraforming you may need to upgrade to paid tier.
There are many more policy tools out there that I haven't even mentioned. It seems like there is a new open source repository every month. There are also some interesting startups that provide sophisticated analysis to help with least privilege access. Check out iamzero if you want a quick way to find the right permissions for a new script. If you have dozens of AWS accounts and are struggling to safely scale access management, K9 Security offers a great tool to help bring order, security and simplicity to chaos.
I’m interested to see how this space develops and would also love to hear your thoughts on any tools you’ve used to take a data-driven approach to least privilege policies.
