# Collect Akamai DataStream 2 logs

**Parser Version:** 2.0
Supported in:

This document explains how to ingest Akamai DataStream 2 logs to Google Security Operations using Amazon S3.

Akamai DataStream 2 captures performance and security logs from the Akamai Intelligent Edge Platform and delivers them in near real-time to a destination of your choice. You can use DataStream 2 to monitor, debug, and analyze CDN and security events across your Akamai properties.

## Before you begin

Make sure you have the following prerequisites:

- A Google SecOps instance.
- Privileged access to **Akamai Control Center** with DataStream 2 configuration permissions.
- Privileged access to **AWS** (S3, IAM, and AWS KMS if the bucket is encrypted with a customer-managed KMS key).

## Configure AWS S3 bucket and IAM for Google SecOps

1. Create **Amazon S3 bucket** following this user guide: [Creating a bucket](https://docs.aws.amazon.com/AmazonS3/latest/userguide/creating-bucket.html).
2. Save bucket **Name** and **Region** for future reference (for example, `akamai-datastream2-logs`).
3. Create a **User** following this user guide: [Creating an IAM user](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_users_create.html#id_users_create_console).
4. Select the created **User**.
5. Select **Security credentials** tab.
6. Click **Create Access Key** in section **Access Keys**.
7. Select **Third-party service** as **Use case**.
8. Click **Next**.
9. Optional: Add description tag.
10. Click **Create access key**.
11. Click **Download .csv file** to save the **Access Key** and **Secret Access Key** for future reference.
12. Click **Done**.
13. Select **Permissions** tab.
14. Click **Add permissions** in section **Permissions policies**.
15. Select **Add permissions**.
16. Select **Attach policies directly**.
17. Search for **AmazonS3FullAccess** policy.
18. Select the policy.
19. Click **Next**.
20. Click **Add permissions**.

## Configure the IAM policy for Akamai S3 uploads

1. In the AWS console, go to **IAM \> Policies \> Create policy \> JSON tab**.
2. Copy and paste the policy below.
3. **Policy JSON** (replace `akamai-datastream2-logs` if you used a different bucket name):

       {
         "Version": "2012-10-17",
         "Statement": [
           {
             "Sid": "AllowAkamaiDataStreamWrite",
             "Effect": "Allow",
             "Action": [
               "s3:PutObject",
               "s3:GetObject",
               "s3:ListBucket"
             ],
             "Resource": [
               "arn:aws:s3:::akamai-datastream2-logs",
               "arn:aws:s3:::akamai-datastream2-logs/*"
             ]
           }
         ]
       }

   > [!NOTE]
   > **Note:** Akamai DataStream 2 requires `s3:PutObject`, `s3:GetObject`, and `s3:ListBucket` permissions to deliver logs and validate the destination.

4. Click **Next**.

5. Enter a policy name (for example, `AkamaiDataStreamS3Policy`).

6. Click **Create policy**.

7. Go to **IAM \> Users \> Create user**.

8. Enter a user name (for example, `akamai-datastream-writer`).

9. Click **Next**.

10. Select **Attach policies directly**.

11. Search for and select the policy created above.

12. Click **Next \> Create user**.

13. Select the created user, go to **Security credentials \> Create access key**.

14. Select **Third-party service** as **Use case** and click **Next \> Create access key**.

15. Click **Download .csv file** to save the **Access Key ID** and **Secret Access Key** for Akamai DataStream 2 configuration.

### Configure AWS KMS permissions for SSE-KMS buckets

Complete this section only if the bucket is encrypted with a customer managed AWS KMS key (SSE-KMS). Without these permissions, Akamai cannot write log files to the bucket and destination validation fails.

1. In the AWS console, go to **S3 \> Buckets** and select your bucket.
2. Go to the **Properties** tab and check **Default encryption** :
   - If **Encryption type** is **Server-side encryption with Amazon S3 managed keys (SSE-S3)** , skip this section and the [KMS section for the Google SecOps user](https://docs.cloud.google.com/chronicle/docs/ingestion/default-parsers/akamai-datastream-2#kms-reader).
   - Otherwise, note the AWS KMS key that the bucket uses.
3. Go to **Key Management Service** and find that key:
   - If the key is listed under **AWS managed keys** (alias `aws/s3`), skip this section and the [KMS section for the Google SecOps user](https://docs.cloud.google.com/chronicle/docs/ingestion/default-parsers/akamai-datastream-2#kms-reader). Principals in your account can use the `aws/s3` key through Amazon S3 without extra permissions.
   - If the key is listed under **Customer managed keys**, select it and copy its ARN.
4. Go to **IAM \> Users** and select `akamai-datastream-writer`.
5. Go to the **Permissions** tab and click **Add permissions \> Create inline policy**.
6. In the **Policy editor** , select **JSON** and paste the following policy:

       {
         "Version": "2012-10-17",
         "Statement": [
           {
             "Sid": "AllowAkamaiDataStreamKmsKeyUse",
             "Effect": "Allow",
             "Action": [
               "kms:GenerateDataKey",
               "kms:Decrypt"
             ],
             "Resource": "KMS_KEY_ARN"
           }
         ]
       }

   Replace `KMS_KEY_ARN` with the key ARN you copied (for example, `arn:aws:kms:us-east-1:111122223333:key/1234abcd-12ab-34cd-56ef-1234567890ab`).

   > [!NOTE]
   > **Note:** `kms:GenerateDataKey` is required to upload encrypted objects. `kms:Decrypt` is required for `s3:GetObject` on encrypted objects, which Akamai requires, and for multipart uploads.

7. Click **Next**.

8. In **Policy name** , enter a name (for example, `akamai-datastream-kms`) and click **Create policy**.

9. Go to **Key Management Service \> Customer managed keys** and select the key.

10. Go to the **Key policy** tab. If the tab shows the default view, click **Switch to policy view**. Check that the key policy allows IAM policies to grant access to the key:

    - If the key policy contains a statement that allows the account root principal (`arn:aws:iam::ACCOUNT_ID:root`) the `kms:*` action, no change is needed. Keys created in the AWS KMS console have this statement by default. The inline policy that you created grants access.
    - Otherwise, click **Edit** and add a statement that allows the user, then click **Save changes**:

        {
          "Sid": "AllowAkamaiDataStreamKmsKeyUse",
          "Effect": "Allow",
          "Principal": {
            "AWS": "arn:aws:iam::ACCOUNT_ID:user/akamai-datastream-writer"
          },
          "Action": [
            "kms:GenerateDataKey",
            "kms:Decrypt"
          ],
          "Resource": "*"
        }

    Replace `ACCOUNT_ID` with your 12-digit AWS account ID. In a key policy, `"Resource": "*"` refers to the key that the policy is attached to.

## Configure Akamai DataStream 2 to deliver logs to Amazon S3

1. Sign in to **Akamai Control Center**.
2. Go to **DataStream 2**.
3. Click **Create a stream**.
4. Select a **log type** appropriate for your property (for example, **Delivery** , **Edge DNS** , **GTM**).
5. In **Data sets**, select the fields you require. Keep defaults unless you have a specific need.
6. Go to **Destination** and select **Amazon S3**.
7. Fill in the S3 destination details:
   - **Name** : Enter a descriptive label for the destination (for example, `SIEM S3 Destination`).
   - **Bucket** : Enter your S3 bucket name (for example, `akamai-datastream2-logs`).
   - **Folder path** : Enter a folder path (for example, `datastream2/logs/`). Supports dynamic variables such as timestamps.
   - **Region**: Select the AWS region that matches your S3 bucket.
   - **Access Key ID**: Enter the access key ID of the IAM user created for Akamai.
   - **Secret Access Key**: Enter the secret access key of the IAM user created for Akamai.
8. Click **Validate \& Save**.

   > [!NOTE]
   > **Note:** Akamai creates a verification file (`Akamai_access_verification_[TimeStamp].txt`) in your S3 bucket to confirm the destination is reachable. If validation fails, verify the IAM permissions and bucket name.

9. Optional: In **Delivery options**, configure the following:

   - **Push frequency** : Select **30 seconds** or **60 seconds**.
   - **Filename prefix** and **suffix** : Customize as needed. Avoid using `.` in prefixes.
10. Click **Next \> Activate**.

## (Optional) Create read-only IAM user for Google SecOps

If you prefer a separate read-only user for Google SecOps to ingest logs:

1. Go to **AWS Console \> IAM \> Policies \> Create policy \> JSON tab**.
2. Copy and paste the policy below (replace `akamai-datastream2-logs` if you used a different bucket name):

       {
         "Version": "2012-10-17",
         "Statement": [
           {
             "Effect": "Allow",
             "Action": [
               "s3:GetObject"
             ],
             "Resource": "arn:aws:s3:::akamai-datastream2-logs/*"
           },
           {
             "Effect": "Allow",
             "Action": [
               "s3:ListBucket"
             ],
             "Resource": "arn:aws:s3:::akamai-datastream2-logs"
           }
         ]
       }

3. Click **Next**.

4. Enter a policy name (for example, `secops-reader-policy`).

5. Click **Create policy**.

6. Go to **IAM \> Users \> Create user**.

7. Enter a username (for example, `secops-reader`).

8. Click **Next**.

9. Select **Attach policies directly** and select the policy created above.

10. Click **Next \> Create user**.

11. Select the created user, go to **Security credentials \> Create access key**.

12. Select **Third-party service** as **Use case** and click **Next \> Create access key**.

13. Click **Download .csv file** to save the **Access Key ID** and **Secret Access Key** for the Google SecOps feed configuration.

## Grant AWS KMS permissions to the Google SecOps user

Complete this section only if the bucket is encrypted with a customer managed AWS KMS key (SSE-KMS), as determined in [Grant AWS KMS permissions for an SSE-KMS bucket](https://docs.cloud.google.com/chronicle/docs/ingestion/default-parsers/akamai-datastream-2#kms-writer). The feed reads the bucket with the access key of one IAM user: `secops-reader` if you created the optional read-only user, or otherwise the user from [Configure AWS S3 bucket and IAM for Google SecOps](https://docs.cloud.google.com/chronicle/docs/ingestion/default-parsers/akamai-datastream-2#configure_aws_s3_bucket_and_iam_for_google_secops). The `AmazonS3FullAccess` policy does not include AWS KMS permissions.

1. Go to **IAM \> Users** and select the user whose access key you use in the feed.
2. Go to the **Permissions** tab and click **Add permissions \> Create inline policy**.
3. In the **Policy editor** , select **JSON** and paste the following policy:

       {
         "Version": "2012-10-17",
         "Statement": [
           {
             "Sid": "AllowSecOpsFeedKmsDecrypt",
             "Effect": "Allow",
             "Action": [
               "kms:Decrypt"
             ],
             "Resource": "KMS_KEY_ARN"
           }
         ]
       }

   Replace `KMS_KEY_ARN` with the ARN of the customer managed key that the bucket uses.

   > [!NOTE]
   > **Note:** The feed only reads objects, so it needs `kms:Decrypt` and does not need `kms:GenerateDataKey`.

4. Click **Next**.

5. In **Policy name** , enter a name (for example, `secops-reader-kms`) and click **Create policy**.

6. If the key policy does not contain the default statement for the account root principal, add a statement for this user as described in [Grant AWS KMS permissions for an SSE-KMS bucket](https://docs.cloud.google.com/chronicle/docs/ingestion/default-parsers/akamai-datastream-2#kms-writer), with the user ARN as `Principal` and only `kms:Decrypt` as `Action`.

## Configure a feed in Google SecOps to ingest Akamai DataStream 2 logs

1. Go to **Settings \> Data Management \> Feeds** . If you are on a SIEM-only platform, go to **Settings \> Feeds**.
2. Click **Add New Feed**.
3. On the next page, click **Configure a single feed**.
4. Enter a unique name for the **Feed name** (for example, `Akamai DataStream 2 Logs`).
5. Select **Amazon S3 V2** as the **Source type**.
6. Select **Akamai DataStream 2** as the **Log type**.
7. Click **Next** and then click **Submit**.
8. Specify values for the following fields:

   - **S3 URI** : `s3://akamai-datastream2-logs/datastream2/logs/`
   - **Source deletion option**: Select the deletion option according to your preference.
   - **Maximum File Age**: Include files modified in the last number of days (default is 180 days).
   - **Access Key ID**: User access key with access to the S3 bucket.
   - **Secret Access Key**: User secret key with access to the S3 bucket.
   - **Asset namespace** : The [asset namespace](https://docs.cloud.google.com/chronicle/docs/investigation/asset-namespaces).
   - **Ingestion labels**: The label to be applied to the events from this feed.

   > [!NOTE]
   > **Note:** If you select the `Delete transferred files` or `Delete transferred files and empty directories` option, make sure that you granted appropriate permissions to the service account.

9. Click **Next** and then click **Submit**.

## UDM mapping table

| Log Field | UDM Mapping | Logic |
|---|---|---|
| `billingRegion_label` | `additional.fields` | Merged |
| `bytes_label` | `additional.fields` | Merged |
| `cache_status_label` | `additional.fields` | Merged |
| `customfield_label` | `additional.fields` | Merged |
| `errorCode_label` | `additional.fields` | Merged |
| `error_code_label` | `additional.fields` | Merged |
| `fileSizeBucket_label` | `additional.fields` | Merged |
| `jsonPayload_cookie_label` | `additional.fields` | Merged |
| `jsonPayload_cp_label` | `additional.fields` | Merged |
| `jsonPayload_overheadbytes_label` | `additional.fields` | Merged |
| `objSize_label` | `additional.fields` | Merged |
| `objsize_label` | `additional.fields` | Merged |
| `overheadBytes_label` | `additional.fields` | Merged |
| `rspContentLen_label` | `additional.fields` | Merged |
| `rspcontentlen_label` | `additional.fields` | Merged |
| `trace_label` | `additional.fields` | Merged |
| `transferTimeMSec_label` | `additional.fields` | Merged |
| `turnAroundTimeMSec_label` | `additional.fields` | Merged |
| `has_principal` | `metadata.event_type` | Mapped: `true` → `NETWORK_HTTP`, `true` → `NETWORK_CONNECTION`, `true` → `STATUS_UPDATE` |
| `has_user` | `metadata.event_type` | Mapped: `true` → `USER_UNCATEGORIZED` |
| `insertId` | `metadata.product_log_id` | Directly mapped |
| `proto_type` | `network.application_protocol` | Directly mapped |
| `jsonPayload.reqmethod` | `network.http.method` | Directly mapped |
| `reqMethod` | `network.http.method` | Directly mapped |
| `jsonPayload_referer` | `network.http.referral_url` | Directly mapped |
| `jsonPayload.statuscode` | `network.http.response_code` | Directly mapped |
| `statusCode` | `network.http.response_code` | Directly mapped |
| `UA` | `network.http.user_agent` | Directly mapped |
| `jsonPayload.ua` | `network.http.user_agent` | Directly mapped |
| `jsonPayload.reqid` | `network.session_id` | Directly mapped |
| `reqId` | `network.session_id` | Directly mapped |
| `jsonPayload.tlsversion` | `network.tls.version` | Directly mapped |
| `tlsVersion` | `network.tls.version` | Directly mapped |
| `version` | `network.tls.version_protocol` | Directly mapped |
| `cliIP` | `principal.asset.ip` | Merged |
| `jsonPayload.cliip` | `principal.asset.ip` | Merged |
| `cliIP` | `principal.ip` | Merged |
| `jsonPayload.cliip` | `principal.ip` | Merged |
| `jsonPayload.city` | `principal.location.city` | Directly mapped |
| `jsonPayload.country` | `principal.location.country_or_region` | Directly mapped |
| `jsonPayload.totalbytes` | `principal.network.received_bytes` | Renamed/mapped |
| `totalBytes` | `principal.network.received_bytes` | Directly mapped |
| `logName` | `security_result.category_details` | Merged |
| `asn_label` | `security_result.detection_fields` | Merged |
| `querystr_label` | `security_result.detection_fields` | Merged |
| `securityrules_label` | `security_result.detection_fields` | Merged |
| `service_name_label` | `security_result.detection_fields` | Merged |
| `jsonPayload.reqhost` | `target.asset.hostname` | Directly mapped |
| `jsonPayload.reqpath` | `target.file.full_path` | Directly mapped |
| `reqPath` | `target.file.full_path` | Directly mapped |
| `jsonPayload.rspcontenttype` | `target.file.mime_type` | Directly mapped |
| `rspContentType` | `target.file.mime_type` | Directly mapped |
| `jsonPayload.reqhost` | `target.hostname` | Directly mapped |
| `jsonPayload.reqport` | `target.port` | Renamed/mapped |
| `reqPort` | `target.port` | Renamed/mapped |
| `uncompressedsize` | `target.process.file.size` | Directly mapped |
| `resource.type` | `target.resource.resource_subtype` | Directly mapped |
| N/A | `metadata.event_type` | Constant: `NETWORK_HTTP` |
| N/A | `metadata.product_name` | Constant: `Akamai Datastream` |
| N/A | `metadata.vendor_name` | Constant: `AkamaiDatastream` |

## Change Log

[View the Change Log for this parser](https://docs.cloud.google.com/chronicle/docs/ingestion/parser-list/akamai-datastream-2-changelog)

**Need more help?** [Get answers from Community members and Google SecOps professionals.](https://security.googlecloudcommunity.com/)