Collect HAProxy logs

Supported in:

Google secops SIEM

This document explains how to ingest HAProxy logs to Google Security Operations using the Bindplane agent.

HAProxy is a high-availability load balancer and proxy server that generates syslog messages for HTTP requests, connection events, backend health checks, and process activity. The parser uses Grok pattern matching to extract fields from various HAProxy log formats and maps them to the Unified Data Model (UDM).

Before you begin

Make sure you have the following prerequisites:

A Google SecOps instance
Windows Server 2016 or later, or Linux host with systemd
Network connectivity between the Bindplane agent and the HAProxy server
If running behind a proxy, ensure firewall ports are open per the Bindplane agent requirements
Administrative access to HAProxy configuration

Get Google SecOps ingestion authentication file

Sign in to the Google SecOps console.
Go to SIEM Settings > Collection Agents.
Download the Ingestion Authentication File.
Save the file securely on the system where the Bindplane agent will be installed.

Note: This JSON file contains credentials for authenticating the Bindplane agent to Google SecOps.

Get Google SecOps customer ID

Sign in to the Google SecOps console.
Go to SIEM Settings > Profile.
Copy and save the Customer ID from the Organization Details section.

Note: The customer ID is required for configuring the Bindplane agent exporter.

Install the Bindplane agent

Install the Bindplane agent on your Windows or Linux operating system according to the following instructions.

Windows installation

Open Command Prompt or PowerShell as an administrator.

Run the following command:

msiexec /i "https://github.com/observIQ/bindplane-agent/releases/latest/download/observiq-otel-collector.msi" /quiet

Wait for the installation to complete.
Verify the installation by running:
```
sc query observiq-otel-collector
```
The service should show as RUNNING.

Linux installation

Open a terminal with root or sudo privileges.

Run the following command:

sudo sh -c "$(curl -fsSlL https://github.com/observiq/bindplane-agent/releases/latest/download/install_unix.sh)" install_unix.sh

Wait for the installation to complete.
Verify the installation by running:
```
sudo systemctl status observiq-otel-collector
```
The service should show as active (running).

Additional installation resources

For additional installation options and troubleshooting, see the Bindplane agent installation guide.

Configure the Bindplane agent to ingest syslog and send to Google SecOps

Locate the configuration file

Linux:

sudo nano /etc/bindplane-agent/config.yaml

Windows:

notepad "C:\Program Files\observIQ OpenTelemetry Collector\config.yaml"

Edit the configuration file

Replace the entire contents of config.yaml with the following configuration:

receivers:
    udplog:
        listen_address: "0.0.0.0:514"

exporters:
    chronicle/haproxy:
        compression: gzip
        creds_file_path: '/etc/bindplane-agent/ingestion-auth.json'
        customer_id: '<customer_id>'
        endpoint: malachiteingestion-pa.googleapis.com
        log_type: HAPROXY
        raw_log_field: body

service:
    pipelines:
        logs/haproxy_to_chronicle:
            receivers:
                - udplog
            exporters:
                - chronicle/haproxy

Configuration parameters

Replace the following placeholders:

Receiver configuration:
- listen_address: IP address and port to listen on:
  - 0.0.0.0 to listen on all interfaces (recommended)
  - Port 514 is the standard syslog port (requires root on Linux; use 1514 for non-root)
Exporter configuration:
- creds_file_path: Full path to ingestion authentication file:
  - Linux: /etc/bindplane-agent/ingestion-auth.json
  - Windows: C:\Program Files\observIQ OpenTelemetry Collector\ingestion-auth.json
- customer_id: Customer ID copied from the Google SecOps console
- endpoint: Regional endpoint URL:
  - US: malachiteingestion-pa.googleapis.com
  - Europe: europe-malachiteingestion-pa.googleapis.com
  - Asia: asia-southeast1-malachiteingestion-pa.googleapis.com
  - See Regional Endpoints for complete list

Save the configuration file

After editing, save the file:
- Linux: Press Ctrl+O, then Enter, then Ctrl+X
- Windows: Click File > Save

Restart the Bindplane agent to apply the changes

To restart the Bindplane agent in Linux, run the following command:

sudo systemctl restart observiq-otel-collector

Verify the service is running:

sudo systemctl status observiq-otel-collector

Check logs for errors:

sudo journalctl -u observiq-otel-collector -f

To restart the Bindplane agent in Windows, choose one of the following options:
- Command Prompt or PowerShell as administrator:
```
net stop observiq-otel-collector && net start observiq-otel-collector
```
- Services console:
  1. Press Win+R, type services.msc, and press Enter.
  2. Locate observIQ OpenTelemetry Collector.
  3. Right-click and select Restart.
  4. Verify the service is running:
```
sc query observiq-otel-collector
```
  5. Check logs for errors:
```
type "C:\Program Files\observIQ OpenTelemetry Collector\log\collector.log"
```

Configure syslog for HAProxy

Sign in to HAProxy using CLI.
Add the log directive in the global section to the Configuration to forward syslog messages over UDP.
- Replace <bindplane-ip> with the actual Bindplane agent IP address.
```
global
    log <bindplane-ip>:514 local0

defaults
    log global
```

UDM mapping table

Log field	UDM mapping	Logic
accept_date_ms
actconn
backend_name
backend_queue
beconn
bytes_read	network.received_bytes	Extracted from `bytes_read` field in the log and converted to an unsigned integer.
captured_request_headers
client_ip	principal.ip	Extracted from `client_ip` field in the log.
client_port	principal.port	Extracted from `client_port` field in the log and converted to an integer.
command_description	metadata.description	Extracted from `command_description` field in the log, if available. Otherwise, it's derived from other fields like `action` or `status` depending on the log message.
datetime	metadata.event_timestamp.seconds	Extracted from `datetime` field in the log, if available. Otherwise, it's derived from the `timestamp` field in the log entry.
description	metadata.description	Extracted from `description` field in the log, if available. Otherwise, it's derived from other fields like `command_description` or `action` depending on the log message.
feconn
frontend_name
http_request	target.url	Extracted from `http_request` field in the log.
http_status_code	network.http.response_code	Extracted from `http_status_code` field in the log and converted to an integer.
http_verb	network.http.method	Extracted from `http_verb` field in the log.
http_version	metadata.product_version	Extracted from `http_version` field in the log and formatted as `HTTP/{version}`.
initiator	target.application	Extracted from `initiator` field in the log.
module
msg	security_result.summary	Extracted from `msg` field in the log.
pid	target.process.pid	Extracted from `pid` field in the log.
process
process_name	target.application	Extracted from `process_name` field in the log.
retries
server_name	target.hostname	Extracted from `server_name` field in the log. If empty, it defaults to the value of `syslog_server`.
severity	security_result.severity	Mapped from `severity` field in the log. `WARNING` maps to `MEDIUM`, `ALERT` maps to `CRITICAL`, and `NOTICE` maps to `INFORMATIONAL`.
shell
srv_queue
srvconn
status
syslog_server	target.hostname, intermediary.hostname	Extracted from `syslog_server` field in the log. Used for both target hostname (if `server_name` is empty) and intermediary hostname.
syslog_timestamp
syslog_timestamp_1
syslog_timestamp_2
syslog_timestamp_4
target_ip
time_backend_connect
time_backend_response
time_duration
time_queue
time_request
timestamp	metadata.event_timestamp.seconds	Extracted from `timestamp` field in the log and parsed for date and time information. Used as the event timestamp.
unknown_parameters1
unknown_parameters2
user_name	target.user.userid	Extracted from `user_name` field in the log.
	metadata.event_type	Set to `NETWORK_HTTP` by default. Changed to specific event types like `PROCESS_UNCATEGORIZED`, `STATUS_UPDATE`, or `USER_UNCATEGORIZED` based on the log message and parsed fields.
	metadata.vendor_name	Set to `HAProxy Enterprise`.
	metadata.product_name	Set to `HAProxy`.
	network.application_protocol	Set to `HTTP` if the `message` field contains `HTTP`.
	metadata.log_type	Set to `HAPROXY`.

Need more help? Get answers from Community members and Google SecOps professionals.