Collect Darktrace logs

Supported in:

This document explains how to ingest Darktrace logs to Google Security Operations using the Bindplane agent.

Darktrace is an AI-powered cybersecurity platform that detects threats and anomalies in real time. The parser handles both CEF and JSON formatted Darktrace logs, extracting fields and mapping them to the Unified Data Model (UDM).

Before you begin

Make sure you have the following prerequisites:

  • A Google SecOps instance
  • Windows Server 2016 or later, or Linux host with systemd
  • Network connectivity between the Bindplane agent and the Darktrace appliance
  • If running behind a proxy, ensure firewall ports are open per the Bindplane agent requirements
  • Privileged access to the Darktrace web UI

Get Google SecOps ingestion authentication file

  1. Sign in to the Google SecOps console.
  2. Go to SIEM Settings > Collection Agents.
  3. Download the Ingestion Authentication File.

  4. Save the file securely on the system where the Bindplane agent will be installed.

Get Google SecOps customer ID

  1. Sign in to the Google SecOps console.
  2. Go to SIEM Settings > Profile.

  3. Copy and save the Customer ID from the Organization Details section.

Install the Bindplane agent

Install the Bindplane agent on your Windows or Linux operating system according to the following instructions.

Windows installation

  1. Open Command Prompt or PowerShell as an administrator.

  2. Run the following command:

    msiexec /i "https://github.com/observIQ/bindplane-agent/releases/latest/download/observiq-otel-collector.msi" /quiet

  3. Wait for the installation to complete.

  4. Verify the installation by running:

    sc query observiq-otel-collector

    The service should show as RUNNING.

Linux installation

  1. Open a terminal with root or sudo privileges.

  2. Run the following command:

    sudo sh -c "$(curl -fsSlL https://github.com/observiq/bindplane-agent/releases/latest/download/install_unix.sh)" install_unix.sh

  3. Wait for the installation to complete.

  4. Verify the installation by running:

    sudo systemctl status observiq-otel-collector

    The service should show as active (running).

Additional installation resources

For additional installation options and troubleshooting, see the Bindplane agent installation guide.

Configure the Bindplane agent to ingest syslog and send to Google SecOps

Locate the configuration file

  • Linux:

    sudo nano /etc/bindplane-agent/config.yaml

  • Windows:

    notepad "C:\Program Files\observIQ OpenTelemetry Collector\config.yaml"

Edit the configuration file

  • Replace the entire contents of config.yaml with the following configuration:

    receivers:
    tcplog:
        listen_address: "0.0.0.0:10282"

exporters:
    chronicle/darktrace:
        compression: gzip
        creds_file_path: '/etc/bindplane-agent/ingestion-auth.json'
        customer_id: '<customer_id>'
        endpoint: malachiteingestion-pa.googleapis.com
        log_type: DARKTRACE
        raw_log_field: body

service:
    pipelines:
        logs/darktrace_to_chronicle:
            receivers:
                - tcplog
            exporters:
                - chronicle/darktrace

Configuration parameters

Replace the following placeholders:

  • Receiver configuration:

    • listen_address: IP address and port to listen on:
      • 0.0.0.0 to listen on all interfaces (recommended)
      • Port 10282 is the default Darktrace CEF syslog port (adjust as needed)

  • Exporter configuration:

    • creds_file_path: Full path to ingestion authentication file:
      • Linux: /etc/bindplane-agent/ingestion-auth.json
      • Windows: C:\Program Files\observIQ OpenTelemetry Collector\ingestion-auth.json
    • customer_id: Customer ID copied from the Google SecOps console
    • endpoint: Regional endpoint URL:
      • US: malachiteingestion-pa.googleapis.com
      • Europe: europe-malachiteingestion-pa.googleapis.com
      • Asia: asia-southeast1-malachiteingestion-pa.googleapis.com
      • See Regional Endpoints for complete list

Save the configuration file

  • After editing, save the file:
    • Linux: Press Ctrl+O, then Enter, then Ctrl+X
    • Windows: Click File > Save

Restart the Bindplane agent to apply the changes

  • To restart the Bindplane agent in Linux, run the following command:

    sudo systemctl restart observiq-otel-collector

    1. Verify the service is running:

      sudo systemctl status observiq-otel-collector

    2. Check logs for errors:

      sudo journalctl -u observiq-otel-collector -f

  • To restart the Bindplane agent in Windows, choose one of the following options:

    • Command Prompt or PowerShell as administrator:

      net stop observiq-otel-collector && net start observiq-otel-collector

    • Services console:

      1. Press Win+R, type services.msc, and press Enter.
      2. Locate observIQ OpenTelemetry Collector.
      3. Right-click and select Restart.

      4. Verify the service is running:

        sc query observiq-otel-collector

      5. Check logs for errors:

        type "C:\Program Files\observIQ OpenTelemetry Collector\log\collector.log"

Configure syslog on Darktrace

  1. Sign in to the Darktrace web UI.
  2. Go to Admin > System Config.
  3. Click Verify Alert Settings.
  4. Provide the following configuration details:
    • CEF Syslog Alerts: Select True.
    • CEF Syslog Server: Enter the Bindplane IP address.
    • CEF Syslog Server Port: Enter the Bindplane port number (for example, 10282).
    • CEF Syslog TCP Alert: Select True.
  5. Click Save.

Supported Darktrace sample logs

  • SYSLOG + KV (CEF)

    {
"acknowledged": false,
"commentCount": 0,
"pbid": 900000001,
"time": 1604055367000,
"creationTime": 1604055367000,
"model": {
    "name": "Compromise::Agent Beacon to New Endpoint",
    "uuid": "dfd6f10b-b91c-4244-9fd5-7c35caf21b33",
    "description": "A device is initiating multiple connections to a new external endpoint...",
    "priority": 2,
    "category": "Informational",
    "mitre": {
    "tactics": [
        "command-and-control"
    ],
    "techniques": [
        "T1071.001"
    ]
    }
},
"triggeredComponents": [
    {
    "time": 1677679818000,
    "ip": " ",
    "port": 443,
    "metric": {
        "name": "externalconnections"
    },
    "triggeredFilters": [
        {
        "filterType": "Connection hostname",
        "trigger": {
            "value": "beacon.external.io"
        }
        }
    ]
    }
],
"score": 0.85,
"device": {
    "did": 90001,
    "macaddress": " ",
    "ip": " ",
    "hostname": "user-laptop-1"
}
}

  • SYSLOG + JSON

    {
"dpi_engine": "dt-sensor-101",
"proto": "tcp",
"source_ip": " ",
"source_port": 51000,
"dest_ip": " ",
"dest_port": 389,
"src": " ",
"dst": " ",
"details": "AP request: srealm is [TESTDOMAIN.LOCAL], service is [LDAP/DC99.testdomain.local/testdomain.local], ST cipher is [aes256-cts-hmac-sha1-96]",
"epochdate": 1690471502.2252,
"@host": "log-collector-host",
"uid": "ABCDEFGHIJ1234567890",
"note": "KERBEROS::App",
"@type": "notice"
}

UDM mapping table

Log field UDM mapping Logic
darktraceUrl security_result.url_back_to_product The value is taken from the darktraceUrl field.
darktrace_host observer.hostname The value is taken from the darktrace_host field if it is not an IP address.
darktrace_ip observer.ip The value is taken from the darktrace_ip field.
darktrace_user observer.user.userid The value is taken from the darktrace_user field.
description security_result.summary, metadata.description The value is taken from the description field.
device.customFields.DT-AUTO.macaddress principal.mac The value is taken from the device.customFields.DT-AUTO.macaddress field.
device.did principal.asset.asset_id The value is taken from the device.did field, converted to a string, and prefixed with Device ID:.
device.firstSeen principal.asset.first_seen_time The value is taken from the device.firstSeen field, converted to a string, and parsed as a UNIX timestamp in milliseconds.
device.hostname principal.hostname, principal.asset.hostname The value is taken from the device.hostname field.
device.ip principal.ip, principal.asset.ip The value is taken from the device.ip field if it matches the IP address format.
device.ips.0.subnet additional.fields.subnet The value is taken from the device.ips.0.subnet field and prefixed with subnet.
device.ips.ip principal.ip, principal.asset.ip The value is taken from the device.ips.ip field for each IP address in the list.
device.lastSeen principal.asset.last_discover_time The value is taken from the device.lastSeen field, converted to a string, and parsed as a UNIX timestamp in milliseconds.
device.macaddress principal.mac The value is taken from the device.macaddress field.
device.objecttype principal.asset.type If the value is device, the UDM field is set to WORKSTATION.
device.sid principal.resource.attribute.labels.sid The value is taken from the device.sid field and converted to a string.
device.typelabel principal.resource.attribute.labels.typelabel The value is taken from the device.typelabel field.
device.typename principal.resource.attribute.labels.typename The value is taken from the device.typename field.
dst target.ip, target.asset.ip The value is taken from the dst field.
dpt target.port The value is taken from the dpt field and converted to an integer.
dvc principal.ip, principal.asset.ip If the value of dvc is an IP address, it is added to the UDM field.
dvchost principal.hostname, principal.asset.hostname The value is taken from the dvchost field.
endpoint target.url The value is taken from the endpoint field.
event_time metadata.event_timestamp The value is taken from the event_time field and parsed as an ISO8601 timestamp.
externalId metadata.product_log_id The value is taken from the externalId field.
incidentEventUrl principal.url The value is taken from the incidentEventUrl field.
ip principal.ip, principal.asset.ip The value is taken from the ip field if it matches the IP address format.
issue_msg security_result.summary The value is taken from the issue_msg field.
message security_result.description The value is taken from the message field.
method network.http.method The value is taken from the method field.
model.description metadata.description The value is taken from the model.description field.
model.name metadata.product_event_type The value is taken from the model.name field.
model.now.category security_result.severity If the value is critical, the UDM field is set to CRITICAL. If the value is Informational, the UDM field is set to INFORMATIONAL. If the value is Suspicious, the UDM field is set to HIGH and the category is set to NETWORK_SUSPICIOUS.
model.now.description metadata.description The value is taken from the model.now.description field.
model.now.message security_result.description The value is taken from the model.now.message field.
model.now.name metadata.product_event_type The value is taken from the model.now.name field.
model.now.pid principal.process.pid The value is taken from the model.now.pid field and converted to a string.
model.now.uuid principal.user.userid The value is taken from the model.now.uuid field and the event type is set to USER_UNCATEGORIZED.
model.pid principal.process.pid The value is taken from the model.pid field and converted to a string.
model.then.description principal.resource.attribute.labels.Model Then Description The value is taken from the model.then.description field.
model.then.name principal.resource.attribute.labels.Model Then Name The value is taken from the model.then.name field.
model.then.pid principal.resource.attribute.labels.Model Then Pid The value is taken from the model.then.pid field and converted to a string.
model.then.uuid principal.resource.attribute.labels.Model Then UUID The value is taken from the model.then.uuid field.
model.uuid principal.user.userid The value is taken from the model.uuid field and the event type is set to USER_UNCATEGORIZED.
relatedBreaches.0.modelName security_result.description The value is taken from the relatedBreaches.0.modelName field.
score security_result.priority, security_result.priority_details If the value is between 0.8 and 1, the priority is set to HIGH_PRIORITY. If the value is between 0.5 and 0.79, the priority is set to MEDIUM_PRIORITY. If the value is between 0 and 0.49, the priority is set to LOW_PRIORITY. The priority details are set to Score : followed by the value of score converted to a string.
severity security_result.severity If the value is 2, the UDM field is set to MEDIUM. If the value is greater than 2, the UDM field is set to HIGH.
shost principal.hostname, principal.asset.hostname The value is taken from the shost field.
smac principal.mac The value is taken from the smac field.
src principal.ip, principal.asset.ip The value is taken from the src field.
status network.http.response_code The value is taken from the status field and converted to a string.
summary metadata.description The value is taken from the summary field.
time The value is taken from the time field, converted to a string, and parsed as a UNIX timestamp in milliseconds.
timestamp The value is taken from the timestamp field and parsed as either an ISO8601 timestamp or a UNIX timestamp in milliseconds.
title security_result.summary The value is taken from the title field.
triggeredComponents.ip intermediary.ip The value is taken from the triggeredComponents.ip field if it matches the IP address format.
triggeredComponents.port intermediary.port The value is taken from the triggeredComponents.port field and converted to an integer.
username principal.user.userid The value is taken from the username field.
metadata.vendor_name Set to DARKTRACE.
metadata.product_name Set to DCIP.
metadata.log_type Set to DARKTRACE.
network.ip_protocol Set to TCP if issue_msg doesn't contain UDP. Otherwise set to UDP.
security_result.action Set to BLOCK if status is 401, otherwise set to ALLOW.
security_result.severity Set to INFORMATIONAL.
network.application_protocol Set to HTTP if method is not empty.
metadata.event_type Set to NETWORK_HTTP if method is not empty. Set to USER_LOGIN if description contains logged into \\\\S+ over ssh. Set to NETWORK_CONNECTION if target_ip is not empty. Otherwise, set to STATUS_UPDATE.
extensions.auth.type Set to MACHINE if description contains logged into \\\\S+ over ssh.
security_result.category Set to DATA_EXFILTRATION if issue_msg contains Exfiltration. Set to NETWORK_MALICIOUS if issue_msg contains Compromise. Otherwise, set to NETWORK_SUSPICIOUS.

Need more help? Get answers from Community members and Google SecOps professionals.