# Collect ESET Threat Intelligence logs

**Parser Version:** 1.0
Supported in:

This document explains how to ingest ESET Threat Intelligence logs into Google Security Operations using Google Cloud Storage V2, a Cloud Run function, and Cloud Scheduler.

ESET Threat Intelligence (ETI) provides evidence-based information and actionable advice about existing or emerging threats. ETI services warn you about malicious software or activity that might threaten your organization or its customers. The service delivers threat intelligence data through TAXII 2.1 feeds in STIX 2.1 format, including APT IoC, botnet C\&C and targets, malicious domains, IPs, URLs, files, phishing URLs, ransomware, and Android threats.

## Before you begin

Make sure you have the following prerequisites:

- A Google SecOps instance
- A Google Cloud project with the following APIs enabled:
  - Cloud Storage API
  - Cloud Run functions API
  - Cloud Scheduler API
  - Cloud Pub/Sub API
- Permissions to create and manage Google Cloud Storage buckets, Cloud Run functions, Pub/Sub topics, and Cloud Scheduler jobs
- Permissions to manage IAM policies on Google Cloud Storage buckets
- An active ESET Threat Intelligence subscription
- Access to the ESET Threat Intelligence portal at https://eti.eset.com

## Create a Google Cloud Storage bucket

1. Go to the [Google Cloud console](https://console.cloud.google.com/).
2. Select your project or create a new one.
3. In the navigation menu, go to **Cloud Storage \> Buckets**.
4. Click **Create bucket**.
5. Provide the following configuration details:

   | Setting | Value |
   |---|---|
   | **Name your bucket** | Enter a globally unique name (for example, `eset-ti-logs`) |
   | **Location type** | Choose based on your needs (Region, Dual-region, Multi-region) |
   | **Location** | Select the location (for example, `us-central1`) |
   | **Storage class** | Standard (recommended for frequently accessed logs) |
   | **Access control** | Uniform (recommended) |
   | **Protection tools** | Optional: Enable object versioning or retention policy |

6. Click **Create**.

## Collect ESET Threat Intelligence TAXII credentials

To enable the Cloud Run function to retrieve threat intelligence data, you need to activate TAXII feeds and generate TAXII credentials from the ETI portal.

### Activate TAXII feeds

1. Sign in to the **ESET Threat Intelligence portal** at https://eti.eset.com.
2. Go to **Data Feeds** in the main menu.
3. Click the **three dots** icon next to the data feed you want to activate.
4. Select **Enable feed**.
5. Repeat steps 3-4 for each feed you want to ingest into Google SecOps.

   > [!NOTE]
   > **Note:** You can activate multiple feeds. Each feed requires a separate Cloud Run function configuration or a single function that iterates over multiple collection IDs.

### Generate TAXII credentials

1. In the ESET Threat Intelligence portal, go to **Admin Settings \> Access Credentials**.
2. Click **Generate TAXII Credentials**.
3. In the dialog that appears, copy and save the following values:

   - **Username**: Your TAXII username
   - **Password**: Your TAXII password

   > [!IMPORTANT]
   > **Important:** Copy and save the username and password immediately. This dialog is the only place where you can access your credentials. If you don't save your credentials or if you lose them, they cannot be recovered. ESET cannot provide you with your credentials. If you lose your credentials, you must delete your TAXII credentials and create new ones.

### Record TAXII feed details

After activating feeds and generating credentials, record the following information for each feed you want to ingest:

1. In the ESET Threat Intelligence portal, go to **Data Feeds**.
2. Click the **three dots** icon next to an activated feed.
3. Select **Show Data Feed detail**.
4. In the side panel, note the following values:

   - **TAXII feed name** : The feed identifier (for example, `botnet stix 2.1`)
   - **TAXII 2 ID** : The collection ID (for example, `0abb06690b0b47e49cd7794396b76b20`)
   - **TAXII 2 feed URL**: The full collection URL

   > [!NOTE]
   > **Note:** The API root URL is the same for all feeds: `https://taxii.eset.com/taxii2/643f4eb5-f8b7-46a3-a606-6d61d5ce223a`

### Available TAXII feeds

- ESET Threat Intelligence provides the following TAXII 2.1 feeds:

  | Feed Name | TAXII Feed Name | Collection ID |
  |---|---|---|
  | **Android infostealer feed** | androidinfostealer stix 2.1 | 9ee501cde0c44d6db4ae995fead1a7c8 |
  | **Android threats feed** | androidthreats stix 2.1 | daf3de8fab144552a1cb5af054ed07ee |
  | **APT IoC** | apt stix 2.1 | 97e3eb74ae5f46dd9e22f677a6938ee7 |
  | **Botnet feed** | botnet stix 2.1 | 0abb06690b0b47e49cd7794396b76b20 |
  | **Botnet - C\&C** | botnet.cc stix 2.1 | d1923a526e8f400dbb301259240ee3d5 |
  | **Botnet - Target** | botnet.target stix 2.1 | 61b6e4f9153e411ca7a9982a2c6ae788 |
  | **Cryptoscam feed** | cryptoscam stix 2.1 | 2c183ce9551a43338c6cc2ed7c2a704d |
  | **Domain feed** | domain stix 2.1 | a34aa0a4f9de419582a883863503f9c4 |
  | **eCrime IoC feed** | ecrime stix 2.1 | 08059376eac84ec4a076cfd682493f91 |
  | **IP feed** | ip stix 2.1 | baaed2a92335418aa753fe944e13c23a |
  | **Malicious email attachments** | emailattachments stix 2.1 | c0d56cf7f81d482eb97fd46beaa4bae0 |
  | **Malicious files feed** | file stix 2.1 | ee6a153ed77e4ec3ab21e76cc2074b9f |
  | **Phishing URL feed** | phishingurl stix 2.1 | d0a6c0f962dd4dd2b3eeb96b18612584 |
  | **PUA adware files feed** | puaadware stix 2.1 | d1bfc81202fc4c6599326771ec2da41d |
  | **PUA dual-use app files feed** | puadualapps stix 2.1 | 970a7d0039ac4668addf058cd9feb953 |
  | **Ransomware feed** | ransomware stix 2.1 | 8d3490d688ce4a989aee9af5c680d8bf |
  | **Scam URL feed** | scamurl stix 2.1 | 2130adc3c67c43f9a3664b187931375e |
  | **Smishing feed** | smishing stix 2.1 | 330ad7d0c736476babe5e49077b96c95 |
  | **SMS scam feed** | smsscam stix 2.1 | 6e20217a2e1246b8ab11be29f759f716 |
  | **URL feed** | url stix 2.1 | 1d3208c143be49da8130f5a66fd3a0fa |

  > [!NOTE]
  > **Note:** All feeds use the same API root URL: `https://taxii.eset.com/taxii2/643f4eb5-f8b7-46a3-a606-6d61d5ce223a`

## Create service account for the Cloud Run function

1. In the **Google Cloud console** , go to **IAM \& Admin \> Service Accounts**.
2. Click **Create Service Account**.
3. Provide the following configuration details:

   - **Service account name** : Enter `eset-ti-collector`
   - **Service account description** : Enter `Service account for ESET Threat Intelligence Cloud Run function to write STIX objects to GCS`
4. Click **Create and Continue**.

5. In the **Grant this service account access to project** section, add the following roles:

   1. Click **Select a role** and search for and select **Storage Object Admin**.
   2. Click **Add Another Role** and search for and select **Cloud Run Invoker**.
6. Click **Continue**.

7. Click **Done**.

### Grant IAM permissions on the Google Cloud Storage bucket

1. Go to **Cloud Storage \> Buckets**.
2. Click on your bucket name (for example, `eset-ti-logs`).
3. Go to the **Permissions** tab.
4. Click **Grant access**.
5. Provide the following configuration details:

   - **Add principals** : Enter the service account email (for example, `eset-ti-collector@PROJECT_ID.iam.gserviceaccount.com`)
   - **Assign roles** : Select **Storage Object Admin**
6. Click **Save**.

## Create Pub/Sub topic

The Pub/Sub topic triggers the Cloud Run function when a message is published by Cloud Scheduler.

1. In the **Google Cloud console** , go to **Pub/Sub \> Topics**.
2. Click **Create Topic**.
3. Provide the following configuration details:
   - **Topic ID** : Enter `eset-ti-trigger`
   - **Add a default subscription**: Leave selected
4. Click **Create**.

## Create the Cloud Run function

1. In the **Google Cloud console** , go to **Cloud Run functions**.
2. Click **Create function**.
3. Provide the following configuration details:

   | Setting | Value |
   |---|---|
   | **Environment** | 2nd gen |
   | **Function name** | `eset-ti-collector` |
   | **Region** | Select the same region as your GCS bucket |
   | **Trigger type** | Cloud Pub/Sub |
   | **Pub/Sub topic** | `eset-ti-trigger` |
   | **Memory allocated** | 512 MiB |
   | **Timeout** | 540 seconds |
   | **Runtime service account** | `eset-ti-collector` |

4. Click **Next**.

5. Set **Runtime** to **Python 3.12**.

6. Set **Entry point** to `main`.

7. In the `requirements.txt` file, add the following dependencies:

       functions-framework==3.*
       google-cloud-storage==2.*
       urllib3==2.*

8. In the `main.py` file, paste the following code:

       import functions_framework
       import json
       import os
       import logging
       import time
       import urllib3
       from datetime import datetime, timedelta, timezone
       from google.cloud import https://docs.cloud.google.com/python/docs/reference/storage/latest

       logger = logging.getLogger(__name__)
       logger.setLevel(logging.INFO)

       HTTP = urllib3.PoolManager(retries=False)
       storage_client = https://docs.cloud.google.com/python/docs/reference/storage/latest.https://docs.cloud.google.com/python/docs/reference/storage/latest/google.cloud.storage.client.Client.html()

       API_ROOT = "https://taxii.eset.com/taxii2/643f4eb5-f8b7-46a3-a606-6d61d5ce223a"
       TAXII_CONTENT_TYPE = "application/taxii+json;version=2.1"

       def _load_state(bucket_name: str, state_key: str, lookback_hours: int) -> str:
           """Return ISO8601 checkpoint (UTC)."""
           try:
               bucket = storage_client.https://docs.cloud.google.com/python/docs/reference/storage/latest/google.cloud.storage.client.Client.html#google_cloud_storage_client_Client_bucket(bucket_name)
               blob = bucket.blob(state_key)
               if blob.exists():
                   state_data = blob.https://docs.cloud.google.com/python/docs/reference/storage/latest/google.cloud.storage.blob.Blob.html#google_cloud_storage_blob_Blob_download_as_text()
                   state = json.loads(state_data)
                   ts = https://docs.cloud.google.com/python/docs/reference/storage/latest/google.cloud.storage.hmac_key.HMACKeyMetadata.html#google_cloud_storage_hmac_key_HMACKeyMetadata_state.get("last_poll_time")
                   if ts:
                       logger.info(f"Loaded state: {ts}")
                       return ts
           except Exception as e:
               logger.warning(f"State read error: {e}")
           default_ts = (
               datetime.now(timezone.utc) - timedelta(hours=lookback_hours)
           ).strftime("%Y-%m-%dT%H:%M:%S.000Z")
           logger.info(f"No previous state found, using lookback: {default_ts}")
           return default_ts

       def _save_state(bucket_name: str, state_key: str, ts: str) -> None:
           """Persist the checkpoint to GCS."""
           bucket = storage_client.https://docs.cloud.google.com/python/docs/reference/storage/latest/google.cloud.storage.client.Client.html#google_cloud_storage_client_Client_bucket(bucket_name)
           blob = bucket.blob(state_key)
           blob.https://docs.cloud.google.com/python/docs/reference/storage/latest/google.cloud.storage.blob.Blob.html#google_cloud_storage_blob_Blob_upload_from_string(
               json.dumps({"last_poll_time": ts}),
               content_type="application/json",
           )
           logger.info(f"Saved state: {ts}")

       def _fetch_objects(
           username: str,
           password: str,
           collection_id: str,
           added_after: str,
           max_records: int,
       ) -> list:
           """Query TAXII 2.1 collection objects with pagination."""
           url = f"{API_ROOT}/collections/{collection_id}/objects/"
           headers = urllib3.make_headers(basic_auth=f"{username}:{password}")
           headers["Accept"] = TAXII_CONTENT_TYPE
           headers["User-Agent"] = "Chronicle-ESET-TI-GCS/1.0"

           all_objects = []
           params = {"added_after": added_after}

           while True:
               qs = "&".join(f"{k}={v}" for k, v in params.items())
               request_url = f"{url}?{qs}" if qs else url

               for attempt in range(3):
                   try:
                       resp = HTTP.request("GET", request_url, headers=headers)
                       break
                   except Exception as e:
                       wait = 2 ** (attempt + 1)
                       logger.warning(f"Request error: {e}, retrying in {wait}s")
                       time.sleep(wait)
               else:
                   raise RuntimeError("Exceeded retry budget for TAXII API")

               if resp.status == 401:
                   raise RuntimeError("Authentication failed: check TAXII credentials")
               if resp.status == 404:
                   raise RuntimeError(
                       f"Collection not found: {collection_id}"
                   )
               if resp.status not in (200, 206):
                   raise RuntimeError(
                       f"TAXII API error {resp.status}: {resp.data[:500]}"
                   )

               body = json.loads(resp.data.decode("utf-8"))
               objects = body.get("objects", [])
               all_objects.extend(objects)
               logger.info(
                   f"Fetched {len(objects)} objects (total: {len(all_objects)})"
               )

               if len(all_objects) >= max_records:
                   logger.info(f"Reached max_records limit: {max_records}")
                   all_objects = all_objects[:max_records]
                   break

               more = body.get("more", False)
               next_param = body.get("next")
               if more and next_param:
                   params = {"added_after": added_after, "next": next_param}
               else:
                   break

           return all_objects

       @functions_framework.cloud_event
       def main(cloud_event):
           """Cloud Run function entry point triggered by Pub/Sub."""
           bucket_name = os.environ["GCS_BUCKET"]
           prefix = os.environ.get("GCS_PREFIX", "eset-ti")
           state_key = os.environ.get("STATE_KEY", "eset-ti/state.json")
           username = os.environ["TAXII_USERNAME"]
           password = os.environ["TAXII_PASSWORD"]
           collection_id = os.environ["COLLECTION_ID"]
           max_records = int(os.environ.get("MAX_RECORDS", "10000"))
           lookback_hours = int(os.environ.get("LOOKBACK_HOURS", "48"))

           try:
               last_poll = _load_state(bucket_name, state_key, lookback_hours)
               objects = _fetch_objects(
                   username, password, collection_id, last_poll, max_records
               )

               if not objects:
                   logger.info("No new STIX objects found")
                   return "No new objects", 200

               now_str = datetime.now(timezone.utc).strftime("%Y%m%d_%H%M%S")
               blob_path = (
                   f"{prefix}/eset_ti_{collection_id}_{now_str}.json"
               )
               ndjson_body = "\n".join(
                   json.dumps(obj, separators=(",", ":")) for obj in objects
               )

               bucket = storage_client.https://docs.cloud.google.com/python/docs/reference/storage/latest/google.cloud.storage.client.Client.html#google_cloud_storage_client_Client_bucket(bucket_name)
               blob = bucket.blob(blob_path)
               blob.https://docs.cloud.google.com/python/docs/reference/storage/latest/google.cloud.storage.blob.Blob.html#google_cloud_storage_blob_Blob_upload_from_string(
                   ndjson_body, content_type="application/x-ndjson"
               )

               new_poll_time = datetime.now(timezone.utc).strftime(
                   "%Y-%m-%dT%H:%M:%S.000Z"
               )
               _save_state(bucket_name, state_key, new_poll_time)

               msg = (
                   f"Wrote {len(objects)} STIX objects to "
                   f"gs://{bucket_name}/{blob_path}"
               )
               logger.info(msg)
               return msg, 200

           except Exception as e:
               logger.error(f"Error collecting ESET TI: {e}")
               raise

9. Click **Deploy**.

10. Wait for the function to be deployed. The status changes to a green checkmark when the deployment is complete.

### Configure environment variables

1. After the function is deployed, go to **Cloud Run functions \> eset-ti-collector**.
2. Click **Edit and deploy new revision**.
3. Click the **Variables and Secrets** tab (or expand **Runtime, build, connections and security settings** for 1st gen).
4. Add the following environment variables:

   | Key | Example value |
   |---|---|
   | `GCS_BUCKET` | `eset-ti-logs` |
   | `GCS_PREFIX` | `eset-ti` |
   | `STATE_KEY` | `eset-ti/state.json` |
   | `TAXII_USERNAME` | Your TAXII username from the ETI portal |
   | `TAXII_PASSWORD` | Your TAXII password from the ETI portal |
   | `COLLECTION_ID` | `0abb06690b0b47e49cd7794396b76b20` |
   | `MAX_RECORDS` | `10000` |
   | `LOOKBACK_HOURS` | `48` |

   > [!NOTE]
   > **Note:** For `COLLECTION_ID`, use the TAXII 2 ID of the specific feed you want to ingest (refer to the Available TAXII feeds table above). To ingest multiple feeds, create a separate Cloud Run function for each collection ID.

5. Click **Deploy**.

## Create Cloud Scheduler job

Cloud Scheduler publishes a message to the Pub/Sub topic on a schedule, triggering the Cloud Run function to poll ESET Threat Intelligence for new STIX objects.

1. In the **Google Cloud console** , go to **Cloud Scheduler**.
2. Click **Create Job**.
3. Provide the following configuration details:

   | Setting | Value |
   |---|---|
   | **Name** | `eset-ti-poll` |
   | **Region** | Select the same region as your function |
   | **Frequency** | `0 */1 * * *` (every hour) |
   | **Timezone** | Select your timezone (for example, `UTC`) |

4. Click **Continue**.

5. In the **Configure the execution** section:

   - **Target type** : Select **Pub/Sub**
   - **Topic** : Select `eset-ti-trigger`
   - **Message body** : Enter `{"poll": true}`
6. Click **Create**.

   > [!NOTE]
   > **Note:** ESET Threat Intelligence has a 14-day data retention window. A polling frequency of 1 hour ensures that no data is missed. You can adjust the cron expression to poll more or less frequently based on your requirements.

### Verify the Cloud Run function

1. In **Cloud Scheduler** , locate the `eset-ti-poll` job.
2. Click **Force Run** to trigger an immediate execution.
3. Go to **Cloud Run functions \> eset-ti-collector \> Logs**.
4. Verify the function executed successfully by checking for log entries such as:

       Fetched 250 objects (total: 250)
       Wrote 250 STIX objects to gs://eset-ti-logs/eset-ti/eset_ti_0abb06690b0b47e49cd7794396b76b20_20250115_103000.json

5. Go to **Cloud Storage \> Buckets \> eset-ti-logs**.

6. Navigate to the `eset-ti/` prefix.

7. Verify that NDJSON files are being created with STIX objects.

## Retrieve the Google SecOps service account and configure the feed

Google SecOps uses a unique service account to read data from your GCS bucket. You must grant this service account access to your bucket.

### Get the service account email

1. Go to **Settings \> Data Management \> Feeds** . If you are on a SIEM-only platform, go to **Settings \> Feeds**.
2. Click **Add New Feed**.
3. Click **Configure a single feed**.
4. In the **Feed name** field, enter a name for the feed (for example, `ESET Threat Intelligence - Botnet`).
5. Select **Google Cloud Storage V2** as the **Source type**.
6. Select **ESET Threat Intelligence** as the **Log type**.
7. Click **Get Service Account**. A unique service account email will be displayed, for example:

       chronicle-12345678@chronicle-gcp-prod.iam.gserviceaccount.com

8. Copy this email address for use in the next step.

   > [!NOTE]
   > **Note:** Each Google SecOps instance has a unique service account. Do not use service accounts from other documentation or examples.

9. Click **Next**.

10. Specify values for the following input parameters:

    - **Storage bucket URL**: Enter the GCS bucket URI with the prefix path:

          gs://eset-ti-logs/eset-ti/

      > [!NOTE]
      > **Note:** Always include the trailing slash (/) at the end of the URI.

    - **Source deletion option**: Select the deletion option according to your preference:

      - **Never**: Never deletes any files after transfers (recommended for testing).
      - **Delete transferred files**: Deletes files after successful transfer.
      - **Delete transferred files and empty directories**: Deletes files and empty directories after successful transfer.

        > [!NOTE]
        > **Note:** If you select a deletion option, the service account must have **Storage Object Admin** role instead of Storage Object Viewer. Update IAM permissions accordingly.

    - **Maximum File Age**: Include files modified in the last number of days (default is 180 days)

    - **Asset namespace** : The [asset namespace](https://docs.cloud.google.com/chronicle/docs/investigation/asset-namespaces)

    - **Ingestion labels** : The label to be applied to the events from this feed (for example, `ESET_IOC`)

11. Click **Next**.

12. Review your new feed configuration in the **Finalize** screen, and then click **Submit**.

## Grant IAM permissions to the Google SecOps service account

The Google SecOps service account needs **Storage Object Viewer** role on your Google Cloud Storage bucket.

1. Go to **Cloud Storage \> Buckets**.
2. Click on your bucket name (for example, `eset-ti-logs`).
3. Go to the **Permissions** tab.
4. Click **Grant access**.
5. Provide the following configuration details:
   - **Add principals**: Paste the Google SecOps service account email
   - **Assign roles** : Select **Storage Object Viewer**
6. Click **Save**.

   > [!NOTE]
   > **Note:** If you plan to use the deletion option "Delete transferred files" or "Delete transferred files and empty directories," grant **Storage Object Admin** role instead of Storage Object Viewer.

## UDM mapping table

| Log Field | UDM Mapping | Logic |
|---|---|---|
| when | metadata.event_timestamp | Timestamp when the event occurred |
|   | metadata.event_type | Type of event (for example, USER_LOGIN, NETWORK_CONNECTION) |
| messageid | metadata.id | Unique identifier for the event |
| protocol | network.ip_protocol | IP protocol (for example, TCP, UDP) |
| deviceName | principal.hostname | Source hostname |
| srcAddr | principal.ip | Source IP address of the connection |
| srcPort | principal.port | Source port number |
| action | security_result.action | Action taken by the security product (for example, ALLOW, BLOCK) |
| dstAddr | target.ip | Destination IP address |
| dstPort | target.port | Destination port number |
|   | metadata.product_name | Product name |
|   | metadata.vendor_name | Vendor/company name |
| `metadata.product_name` | `ESET_IOC` | Mapped from changelog |
| `confidence` | `event.ioc.raw_severity` | Mapped from changelog |

## Change Log

[View the Change Log for this parser](https://docs.cloud.google.com/chronicle/docs/ingestion/parser-list/eset-ioc-changelog)

**Need more help?** [Get answers from Community members and Google SecOps professionals.](https://security.googlecloudcommunity.com/)