Collect AWS IAM logs

Supported in:

Google secops SIEM

Note: This feature is covered by Pre-GA Offerings Terms of the Google Security Operations Service Specific Terms. Pre-GA features might have limited support, and changes to pre-GA features might not be compatible with other pre-GA versions. For more information, see the Google SecOps Technical Support Service guidelines and the Google SecOps Service Specific Terms.

This document explains how to ingest AWS IAM logs to Google Security Operations. The parser transforms raw JSON formatted logs into a structured Unified Data Model (UDM). It extracts relevant fields like user details, role information, permissions, and timestamps, mapping them to corresponding UDM fields for consistent security analysis.

Before you begin

Ensure that you have the following prerequisites:

• A Google SecOps instance
• Privileged access to AWS console
• Permissions to create IAM users, roles, and policies

Choose your integration method

Google SecOps supports two methods for ingesting AWS IAM data:

Method A: CloudTrail + Amazon S3 (Activity logs)

• What it collects: IAM activity logs (who performed which actions)
• Data source: AWS CloudTrail events
• Latency: Several minutes (polling-based)
• Use case: Historical audit trail, compliance reporting
• Feed source type: Amazon S3 V2

Method B: Third party API (Configuration snapshot)

• What it collects: IAM configuration data (users, groups, roles, policies)
• Data source: AWS IAM API direct calls
• Latency: Near real-time (periodic polling)
• Use case: Real-time IAM configuration monitoring, access review
• Feed source type: Third party API

Note: For comprehensive coverage, you can use both methods simultaneously. CloudTrail captures actions taken, while the Third party API method captures the current state of IAM entities.

Method A: CloudTrail + Amazon S3 integration

This method uses AWS CloudTrail to capture IAM activity and stores logs in Amazon S3, which Google SecOps then ingests.

Create an Amazon S3 bucket

1. Create an Amazon S3 bucket following this user guide: Creating a bucket.
2. Save bucket Name and Region for future reference (for example, iam-activity-logs-bucket).

Configure S3 bucket policy for CloudTrail

CloudTrail needs permissions to write logs to your S3 bucket.

1. In the Amazon S3 console, select your bucket.
2. Go to Permissions > Bucket policy.

3. Click Edit and add the following policy:

   {
     "Version": "2012-10-17",
     "Statement": [
       {
         "Sid": "CloudTrailAclCheck",
         "Effect": "Allow",
         "Principal": {
           "Service": "cloudtrail.amazonaws.com"
         },
         "Action": "s3:GetBucketAcl",
         "Resource": "arn:aws:s3:::iam-activity-logs-bucket"
       },
       {
         "Sid": "CloudTrailWrite",
         "Effect": "Allow",
         "Principal": {
           "Service": "cloudtrail.amazonaws.com"
         },
         "Action": "s3:PutObject",
         "Resource": "arn:aws:s3:::iam-activity-logs-bucket/AWSLogs/*",
         "Condition": {
           "StringEquals": {
             "s3:x-amz-acl": "bucket-owner-full-control"
           }
         }
       }
     ]
   }
   

   • Replace iam-activity-logs-bucket with your actual bucket name.

4. Click Save changes.

Configure CloudTrail to capture IAM activity

1. Sign in to the AWS Management Console.
2. In the search bar, type and select CloudTrail from the services list.
3. Click Create trail.
4. Provide the following configuration details:
   • Trail name: Enter a descriptive name (for example, IAMActivityTrail).
   • Apply trail to all regions: Select Yes to capture activities across all regions.
   • Storage location: Select Use existing S3 bucket and choose the bucket created earlier.
   • Log file prefix (optional): Enter a prefix (for example, iam-logs/).
   • Log file SSE-KMS encryption: Optional. If enabled, create or select a KMS key.
5. Click Next.
6. Configure event selection:
   • Management events: Select Read and Write to capture both read and write events on IAM resources.
   • Data events: Optional. Enable S3 and Lambda data events if needed.
   • Insights events: Optional. Enable for unusual activity detection.
7. Click Next.
8. Review the configuration and click Create trail.

Note: CloudTrail begins logging IAM activity immediately. Logs are delivered to S3 within approximately 15 minutes.

Optional: Enable S3 bucket versioning (recommended)

1. In the Amazon S3 console, select your bucket.
2. Go to Properties > Bucket Versioning.
3. Click Edit.
4. Select Enable.
5. Click Save changes.

Note: Versioning ensures that logs are preserved even if accidentally deleted or overwritten.

Create IAM user for Google SecOps S3 access

1. Create a User following this user guide: Creating an IAM user.
2. Select the created User.
3. Select the Security credentials tab.
4. Click Create Access Key in section Access Keys.
5. Select Third-party service as Use case.
6. Click Next.
7. Optional: Add a description tag.
8. Click Create access key.
9. Click Download CSV file to save the Access Key and Secret Access Key for future reference.
10. Click Done.
11. Select the Permissions tab.
12. Click Add permissions in section Permissions policies.
13. Select Add permissions.
14. Select Attach policies directly.
15. Click Create policy in a new tab.
16. In the Policy editor, select JSON tab.

17. Enter the following policy:

    {
      "Version": "2012-10-17",
      "Statement": [
        {
          "Effect": "Allow",
          "Action": [
            "s3:GetObject",
            "s3:ListBucket"
          ],
          "Resource": [
            "arn:aws:s3:::iam-activity-logs-bucket",
            "arn:aws:s3:::iam-activity-logs-bucket/*"
          ]
        }
      ]
    }
    

    • Replace iam-activity-logs-bucket with your actual bucket name.

18. Click Next.

19. Name the policy chronicle-s3-read-policy.

20. Click Create policy.

21. Return to the user creation tab and refresh the policies list.

22. Search for and select chronicle-s3-read-policy.

23. Click Next.

24. Click Add permissions.

Optional: Create the Lambda function for real-time export

If you need near real-time export of CloudTrail logs to S3:

1. In the AWS Console, go to Lambda > Functions > Create function.
2. Click Author from scratch.

3. Provide the following configuration details:

   Setting	Value
   Name	ExportIAMLogsToS3
   Runtime	Python 3.13
   Architecture	x86_64
   Execution role	Create a new role with basic Lambda permissions

4. Click Create function.

5. After the function is created, open the Code tab, delete the stub and enter the following code:

   import boto3
   import gzip
   from io import BytesIO
   
   s3 = boto3.client('s3')
   logs = boto3.client('logs')
   
   def lambda_handler(event, context):
       log_group = event['logGroup']
       log_stream = event['logStream']
   
       log_events = logs.get_log_events(
           logGroupName=log_group,
           logStreamName=log_stream,
           startFromHead=True
       )
   
       log_data = "\n".join([event['message'] for event in log_events['events']])
   
       # Compress and upload to S3
       compressed_data = gzip.compress(log_data.encode('utf-8'))
       s3.put_object(
           Bucket='iam-activity-logs-bucket',
           Key=f'iam-logs/{log_stream}.gz',
           Body=compressed_data
       )
   
       return {
           'statusCode': 200,
           'body': 'Logs exported successfully'
       }
   

   • Replace iam-activity-logs-bucket with your bucket name.

6. Click Deploy to save the function code.

Optional: Configure Lambda execution role permissions

1. In the same function, select the Configuration tab.
2. Select Permissions from the left menu.
3. Click the Execution role name to open it in IAM console.
4. Click Add permissions > Attach policies.
5. Click Create policy in a new tab.

6. Select JSON tab and paste the following policy:

   {
     "Version": "2012-10-17",
     "Statement": [
       {
         "Effect": "Allow",
         "Action": [
           "logs:GetLogEvents",
           "logs:FilterLogEvents",
           "logs:DescribeLogGroups",
           "logs:DescribeLogStreams"
         ],
         "Resource": "*"
       },
       {
         "Effect": "Allow",
         "Action": [
           "s3:PutObject"
         ],
         "Resource": "arn:aws:s3:::iam-activity-logs-bucket/*"
       }
     ]
   }
   

   • Replace iam-activity-logs-bucket with your bucket name.

7. Click Next.

8. Name the policy lambda-iam-logs-export-policy.

9. Click Create policy.

10. Return to the role tab and refresh.

11. Search for and select lambda-iam-logs-export-policy.

12. Click Add permissions.

Optional: Configure Lambda timeout

1. In the Lambda function, stay on the Configuration tab.
2. Select General configuration from the left menu.
3. Click Edit.
4. Change Timeout to 5 minutes (300 seconds).
5. Click Save.

Note: Timeout is a hard upper limit, meaning if the function is still running when the timer hits 5 minutes, AWS Lambda will terminate it. Any shorter execution just finishes normally; you are billed only for the actual run time.

Optional: Configure Lambda trigger for CloudWatch Logs

1. In the Lambda function, select the Function overview section at the top.
2. Click Add trigger.
3. In the Trigger configuration drop-down, select CloudWatch Logs.
4. Provide the following configuration details:
   • Log group: Select or enter the CloudWatch Logs log group associated with CloudTrail (for example, /aws/cloudtrail/).
   • Filter name: Enter a descriptive name (for example, IAM-events-filter).
   • Filter pattern: Leave empty to capture all events, or enter a specific pattern.
5. Click Add.

Set up feeds

There are two different entry points to set up feeds in the Google SecOps platform:

• SIEM Settings > Feeds > Add New Feed
• Content Hub > Content Packs > Get Started

How to set up the AWS IAM feed

1. Click the Amazon Cloud Platform pack.
2. Locate the AWS IAM log type.

3. Specify the values in the following fields.

   • Source Type: Third party API
   • Username: Username to authenticate with
   • Secret: Secret to authenticate with

   Advanced options

   • Feed Name: A prepopulated value that identifies the feed.
   • Asset Namespace: Namespace associated with the feed.
   • Ingestion Labels: Labels applied to all events from this feed.

4. Click Create feed.

For more information about configuring multiple feeds for different log types within this product family, see Configure feeds by product.

Using SIEM Settings

1. Go to SIEM Settings > Feeds.
2. Click Add New Feed.
3. On the next page, click Configure a single feed.
4. In the Feed name field, enter a name for the feed (for example, AWS IAM CloudTrail Logs).
5. Select Amazon S3 V2 as the Source type.
6. Select AWS IAM as the Log type.
7. Click Next.

8. Specify values for the following input parameters:

   • S3 URI: s3://iam-activity-logs-bucket/

     • Replace iam-activity-logs-bucket with your actual bucket name.
     Note: Include the trailing slash (/) at the end of the URI.

   • Source deletion option: Select deletion option according to your preference:

     • Never: Never deletes any files after transfers (recommended for testing).
     • On success: Deletes all files and empty directories after successful transfer (for cost optimization).
     Note: If you select deletion option, ensure the IAM user has s3:DeleteObject permission.

   • Maximum File Age: Include files modified in the last number of days. Default is 180 days.

   • Access Key ID: User access key with access to the S3 bucket.

   • Secret Access Key: User secret key with access to the S3 bucket.

   • Asset namespace: The asset namespace.

   • Ingestion labels: The label to be applied to the events from this feed.

9. Click Next.

10. Review your new feed configuration in the Finalize screen, and then click Submit.

Method B: Third party API integration

This method uses direct AWS IAM API calls to collect current IAM configuration data (users, groups, roles, policies).

Get Google SecOps IP ranges

1. Go to SIEM Settings > Feeds.
2. Click Add New Feed.
3. Note the IP ranges displayed at the top of the page.
4. Alternatively, retrieve IP ranges programmatically using the Feed Management API.

Create IAM user with required permissions

Important: To allow Google SecOps to retrieve AWS IAM data via API, you must create an IAM user with appropriate read permissions.

Option A: Use AWS Managed Policy (Recommended)

1. Create a User following this user guide: Creating an IAM user.
2. Select the created User.
3. Select the Security credentials tab.
4. Click Create Access Key in section Access Keys.
5. Select Third-party service as Use case.
6. Click Next.
7. Optional: Add a description tag.
8. Click Create access key.
9. Click Download CSV file to save the Access Key and Secret Access Key for future reference.
10. Click Done.
11. Select the Permissions tab.
12. Click Add permissions in section Permissions policies.
13. Select Add permissions.
14. Select Attach policies directly.
15. Search for IAMReadOnlyAccess (AWS managed policy).
16. Select the policy.
17. Click Next.
18. Click Add permissions.

Important: The IAMReadOnlyAccess policy includes all required permissions:

• iam:GetUser
• iam:ListUsers
• iam:GetGroup
• iam:ListGroups
• iam:GetPolicy
• iam:ListPolicies
• iam:GetRole
• iam:ListRoles
• iam:ListAttachedUserPolicies
• iam:ListAttachedGroupPolicies
• iam:ListAttachedRolePolicies
• iam:GetAccountSummary

Option B: Create Custom Policy (Least Privilege)

If your security policy requires minimal permissions instead of the managed policy:

1. In the AWS console, go to IAM > Policies > Create policy > JSON tab.

2. Paste the following policy:

   {
     "Version": "2012-10-17",
     "Statement": [
       {
         "Effect": "Allow",
         "Action": [
           "iam:GetUser",
           "iam:ListUsers",
           "iam:GetGroup",
           "iam:ListGroups",
           "iam:GetPolicy",
           "iam:ListPolicies",
           "iam:GetRole",
           "iam:ListRoles",
           "iam:ListAttachedUserPolicies",
           "iam:ListAttachedGroupPolicies",
           "iam:ListAttachedRolePolicies",
           "iam:GetAccountSummary"
         ],
         "Resource": "*"
       }
     ]
   }
   

3. Click Next.

4. Name the policy chronicle-iam-api-read-policy.

5. Click Create policy.

6. Create a User following this user guide: Creating an IAM user.

7. Select the created User.

8. Select the Security credentials tab.

9. Click Create Access Key in section Access Keys.

10. Select Third-party service as Use case.

11. Click Next.

12. Optional: Add a description tag.

13. Click Create access key.

14. Click Download CSV file to save the Access Key and Secret Access Key for future reference.

15. Click Done.

16. Select the Permissions tab.

17. Click Add permissions in section Permissions policies.

18. Select Add permissions.

19. Select Attach policies directly.

20. Search for and select chronicle-iam-api-read-policy.

21. Click Next.

22. Click Add permissions.

Note: All actions are read-only and don't modify AWS resources.

Configure a feed in Google SecOps to ingest IAM configuration data

Using Content Hub (recommended)

Note: The Content Hub is not available on the SIEM standalone platform. To upgrade, contact your Google SecOps representative.

1. Go to Content Hub > Content Packs > Get Started.
2. Click the Amazon Cloud Platform pack.
3. Locate the AWS IAM log type.
4. Select Third party API from the Source Type drop-down.
5. Provide the following configuration details:
   • Username: The Access Key ID from the IAM user created earlier.
   • Secret: The Secret Access Key from the IAM user created earlier.
   • Feed Name: A prepopulated value that identifies the feed (for example, AWS IAM API Configuration).
   • Asset Namespace: Namespace associated with the feed.
   • Ingestion Labels: Labels applied to all events from this feed.
6. Click Create feed.

For more information about configuring multiple feeds for different log types within this product family, see Configure feeds by product.

Using SIEM Settings

1. Go to SIEM Settings > Feeds.
2. Click Add New Feed.
3. On the next page, click Configure a single feed.
4. In the Feed name field, enter a name for the feed (for example, AWS IAM API Configuration).
5. Select Third party API as the Source type.
6. Select AWS IAM as the Log type.
7. Click Next.

8. Specify values for the following input parameters:

   • Username: The Access Key ID from the IAM user created earlier.
   • Secret: The Secret Access Key from the IAM user created earlier.
   • Region: The AWS region (for example, us-east-1).
   Note: AWS IAM is a global service, but you must specify a region for API endpoint routing.
   • Asset namespace: The asset namespace.
   • Ingestion labels: The label to be applied to the events from this feed.

9. Click Next.

10. Review your new feed configuration in the Finalize screen, and then click Submit.

UDM mapping table

Log Field	UDM Mapping	Logic
Arn	entity.entity.resource.name	Directly mapped from the ARN field. Applied to various entity types. For GROUP entity type, mapped from Group.Arn.
AssumeRolePolicyDocument	entity.entity.resource.attribute.permissions.name	Directly mapped from the AssumeRolePolicyDocument field, but only for RESOURCE entity type.
CreateDate	entity.entity.user.attribute.creation_time	Directly mapped from the CreateDate field and converted to Chronicle's timestamp format.
CreateDate	entity.entity.resource.attribute.creation_time	Directly mapped from the CreateDate field and converted to Chronicle's timestamp format.
Group.Arn	entity.entity.resource.name	Directly mapped from the Group.Arn field.
Group.CreateDate	entity.entity.group.attribute.creation_time	Directly mapped from the Group.CreateDate field and converted to Chronicle's timestamp format.
Group.GroupID	entity.entity.group.product_object_id	Directly mapped from the Group.GroupID field.
Group.GroupName	entity.entity.group.group_display_name	Directly mapped from the Group.GroupName field.
Group.GroupName	entity.entity.group.email_addresses	Directly mapped from the Group.GroupName field.
Group.Path	entity.entity.group.attribute.labels.value	Directly mapped from the Group.Path field, the key is hardcoded to path.
IsTruncated	entity.entity.group.attribute.labels.value	Directly mapped from the IsTruncated field converted to string, the key is hardcoded to is_truncated.
Marker	entity.entity.group.attribute.labels.value	Directly mapped from the Marker field, the key is hardcoded to marker.
PasswordLastUsed	entity.entity.user.last_login_time	Directly mapped from the PasswordLastUsed field and converted to Chronicle's timestamp format.
Path	entity.entity.user.attribute.labels.value	Directly mapped from the Path field for USER entity type, key is hardcoded to path.
Path	entity.entity.resource.attribute.labels.value	Directly mapped from the Path field for RESOURCE entity type, the key is hardcoded to path.
PermissionsBoundary.PermissionsBoundaryArn	entity.entity.resource.attribute.labels.value	Directly mapped from PermissionsBoundary.PermissionsBoundaryArn field, the key is hardcoded to permissions_boundary_arn.
PermissionsBoundary.PermissionsBoundaryType	entity.entity.resource.attribute.labels.value	Directly mapped from PermissionsBoundary.PermissionsBoundaryType field, the key is hardcoded to permissions_boundary_type.
RoleID	entity.entity.resource.product_object_id	Directly mapped from the RoleID field.
RoleLastUsed.LastUsedDate	entity.entity.resource.attribute.labels.value	Directly mapped from RoleLastUsed.LastUsedDate field, the key is hardcoded to role_last_used_date.
RoleLastUsed.Region	entity.entity.location.name	Directly mapped from the RoleLastUsed.Region field.
RoleName	entity.entity.resource.attribute.roles.name	Directly mapped from the RoleName field.
Tags.Key	entity.entity.user.attribute.labels.key	Used as the key for the labels in the user entity.
Tags.Value	entity.entity.user.attribute.labels.value	Used as the value for the labels in the user entity.
UserID	entity.entity.user.product_object_id	Directly mapped from the UserID field.
UserName	entity.entity.user.userid	Directly mapped from the UserName field.
Users.Arn	relations.entity.resource.name	Directly mapped from Users.Arn field within the user relation.
Users.CreateDate	relations.entity.user.attribute.creation_time	Directly mapped from Users.CreateDate field within the user relation and converted to Chronicle's timestamp format.
Users.PasswordLastUsed	relations.entity.user.last_login_time	Directly mapped from Users.PasswordLastUsed field within the user relation and converted to Chronicle's timestamp format.
Users.Path	relations.entity.user.attribute.labels.value	Directly mapped from Users.Path field within the user relation, the key is hardcoded to path.
Users.PermissionsBoundary.PermissionsBoundaryArn	relations.entity.resource.attribute.labels.value	Directly mapped from Users.PermissionsBoundary.PermissionsBoundaryArn field within the user relation, the key is hardcoded to permissions_boundary_arn.
Users.PermissionsBoundary.PermissionsBoundaryType	relations.entity.resource.attribute.labels.value	Directly mapped from Users.PermissionsBoundary.PermissionsBoundaryType field within the user relation, the key is hardcoded to permissions_boundary_type.
Users.UserID	relations.entity.user.product_object_id	Directly mapped from Users.UserID field within the user relation.
Users.UserName	relations.entity.user.userid	Directly mapped from Users.UserName field within the user relation.
N/A	entity.metadata.collected_timestamp	Populated with the event ingestion timestamp.
N/A	entity.metadata.vendor_name	Hardcoded to AWS.
N/A	entity.metadata.product_name	Hardcoded to AWS IAM.
N/A	entity.metadata.entity_type	Determined based on the presence of specific fields: USER if UserID exists, RESOURCE if RoleID exists, GROUP if Group.GroupName exists.
N/A	entity.entity.resource.resource_subtype	Set to User for USER and Role for RESOURCE entity types.
N/A	entity.entity.resource.resource_type	Set to ACCESS_POLICY for RESOURCE entity type.
N/A	entity.entity.resource.attribute.cloud.environment	Hardcoded to AMAZON_WEB_SERVICES.
N/A	relations.entity_type	Hardcoded to USER for user relations.
N/A	relations.relationship	Hardcoded to MEMBER for user group relations.
N/A	relations.direction	Hardcoded to UNIDIRECTIONAL.
N/A	relations.entity.resource.resource_subtype	Hardcoded to User for user relations.

Need more help? Get answers from Community members and Google SecOps professionals.