Collect Linux auditd and AIX systems logs

Supported in:

Google secops SIEM

Note: This feature is covered by Pre-GA Offerings Terms of the Google Security Operations Service Specific Terms. Pre-GA features might have limited support, and changes to pre-GA features might not be compatible with other pre-GA versions. For more information, see the Google SecOps Technical Support Service guidelines and the Google SecOps Service Specific Terms.

This parser handles Linux audit logs in SYSLOG format, transforming them into UDM. It processes both JSON-formatted and plain text log messages, extracting fields using grok, XML parsing, and JSON parsing techniques, and mapping them to appropriate UDM fields based on the event type. The parser also handles specific audit log formats from AIX systems and enriches the UDM with additional fields like security_result and intermediary details.

Before you begin

• Ensure that you have a Google Security Operations instance.
• Ensure that you have root access to the Auditd host.
• Ensure that you installed rsyslog on the Auditd host.
• Ensure that you have a Windows 2012 SP2 or later or Linux host with systemd.
• If running behind a proxy, ensure that the firewall ports are open.

Get Google SecOps ingestion authentication file

1. Sign in to the Google SecOps console.
2. Go to SIEM Settings > Collection Agents.
3. Download the Ingestion Authentication File.

Get Google SecOps customer ID

1. Sign in to the Google SecOps console.
2. Go to SIEM Settings > Profile.
3. Copy and save the Customer ID from the Organization Details section.

Install Bindplane Agent

1. For Windows installation, run the following script:
   msiexec /i "https://github.com/observIQ/bindplane-agent/releases/latest/download/observiq-otel-collector.msi" /quiet
2. For Linux installation, run the following script:
   sudo sh -c "$(curl -fsSlL https://github.com/observiq/bindplane-agent/releases/latest/download/install_unix.sh)" install_unix.sh
3. Additional installation options can be found in this installation guide.

Configure Bindplane Agent to ingest Syslog and send to Google SecOps

1. Access the machine where Bindplane is installed.

2. Edit the config.yaml file as follows:

   receivers:
     tcplog:
       # Replace the below port <54525> and IP <0.0.0.0> with your specific values
       listen_address: "0.0.0.0:54525" 
   
   exporters:
       chronicle/chronicle_w_labels:
           compression: gzip
           # Adjust the creds location below according the placement of the credentials file you downloaded
           creds: '{ json file for creds }'
           # Replace <customer_id> below with your actual ID that you copied
           customer_id: <customer_id>
           endpoint: malachiteingestion-pa.googleapis.com
           # You can apply ingestion labels below as preferred
           ingestion_labels:
           log_type: SYSLOG
           namespace: auditd
           raw_log_field: body
   service:
       pipelines:
           logs/source0__chronicle_w_labels-0:
               receivers:
                   - tcplog
               exporters:
                   - chronicle/chronicle_w_labels
   

3. Restart Bindplane Agent to apply the changes using the following command: sudo systemctl bindplane restart

Exporting Syslog from Auditd

1. Access the machine from which you want to export audit logs.

2. Open the Auditd configuration file (typically located at /etc/audit/auditd.conf).

   sudo vi /etc/audit/auditd.conf
   

3. Find or add the following line(s) to configure auditd:

   active = yes
   output = syslog
   log_format = ENRICHED
   dispatcher = /sbin/audispd
   

Optional: Specify Syslog Facility: Add or modify the following line in auditd.conf:

```none
syslog_facility = LOG_AUTHPRIV
```

Note: Common facilities include LOG_AUTHPRIV (for security/authorization messages), LOG_LOCAL0 through LOG_LOCAL7 (for custom applications).

1. Open audispd configuration file (typically located at /etc/audisp/plugins.d/syslog.conf):

   sudo vi /etc/audisp/plugins.d/syslog.conf
   

2. Find or add the following line(s) to configure audispd:

   active = yes
   direction = out
   path = builtin_syslog
   type = builtin
   args = LOG_INFO
   format = string
   

3. Restart Auditd service to apply the changes:

   sudo systemctl restart auditd
   

4. Use a tool like tail to monitor the syslog and verify that Auditd logs are being sent:

   tail -f /var/log/syslog | grep auditd # Follow syslog and filter for auditd messages (path may vary depending on your system)
   

5. Edit the rsyslog.conf or create a custom configuration:

   sudo vi /etc/rsyslog.d/50-audit-forwarding.conf
   

6. Add a rule to forward logs:

   • if $programname == 'auditd' then @@<Bindplane_Agent>:<Bindplane_Port>
   • Use @for UDP or @@ for TCP
   • Replace <BindPlane_Agent> with your server's IP/hostname.
   • Replace <BindPlane_Port> with your server's Port.

7. Restart rsyslog service to apply the changes:

   sudo systemctl restart rsyslog
   

UDM mapping table

Log field	UDM mapping	Remark
acct	target.user.user_display_name	The value of acct from the raw log is mapped to the target.user.user_display_name field in the UDM. This represents the account associated with the event.
addr	principal.ip	The value of addr from the raw log is mapped to the principal.ip field in the UDM. This represents the IP address of the principal involved in the event.
additional.fields	additional.fields	Additional fields from parsed key-value pairs or labels are added to the additional.fields array in the UDM.
agent.googleapis.com/log_file_path	(Not Mapped)	This label is present in some raw logs but is not mapped to the IDM object in the UDM.
algo	(Not used in this example)	Although present in the parser and some raw logs, this field isn't used in the provided example and doesn't appear in the final UDM.
application	principal.application	Derived from the terminal field in the raw log or other fields like exe depending on the log type. Represents the application involved.
arch	security_result.about.platform_version	The architecture from the raw log's arch field is mapped to security_result.about.platform_version.
auid	about.user.userid, security_result.detection_fields.auid	The audit user ID (auid) is mapped to about.user.userid and added as a detection field in security_result.
cmd	target.process.command_line	The command from the raw log's cmd field is mapped to target.process.command_line.
collection_time	(Not Mapped)	This field is the log collection time and is not mapped to the IDM object in the UDM.
comm	principal.application	The command name (comm) is mapped to principal.application.
COMMAND	target.process.command_line
compute.googleapis.com/resource_name	principal.hostname	The resource name from this label is mapped to principal.hostname.
create_time	(Not Mapped)	This field is not mapped to the IDM object in the UDM.
cwd	security_result.detection_fields.cwd	The current working directory (cwd) is added as a detection field in security_result.
data	(Processed)	The data field contains the main log message and is processed by the parser to extract various fields. It is not directly mapped to a single UDM field.
exe	target.process.file.full_path	The executable path (exe) is mapped to target.process.file.full_path.
extensions.auth.type	extensions.auth.type	The authentication type is set by the parser logic based on the event type. Often set to MACHINE or AUTHTYPE_UNSPECIFIED.
fp	network.tls.client.certificate.sha256	The fingerprint (fp) is parsed to extract the SHA256 hash and mapped to network.tls.client.certificate.sha256.
_Item_Id	metadata.product_log_id
insertId	(Not Mapped)	This field is not mapped to the IDM object in the UDM.
jsonPayload.message	(Processed)	This field contains the main log message in JSON format and is processed by the parser.
key	security_result.about.registry.registry_key	The key field is mapped to security_result.about.registry.registry_key.
labels	(Processed)	Labels from the raw log are processed and mapped to various UDM fields or added to additional.fields.
logName	(Not Mapped)	This field is not mapped to the IDM object in the UDM.
metadata.product_event_type	SECCOMP	The key exchange curve is extracted from the raw log and mapped to this field.
msg	security_result.summary	The message (msg) is often used to populate the security_result.summary field.
network.application_protocol	network.application_protocol	Set by the parser logic based on the event type (e.g., SSH, HTTP).
network.direction	network.direction	Set by the parser logic based on the event type (e.g., INBOUND, OUTBOUND).
network.ip_protocol	network.ip_protocol	Set by the parser logic, usually to TCP for SSH events.
network.session_id	network.session_id	Mapped from the ses field or derived from other fields.
network.tls.cipher	network.tls.cipher	The cipher information is extracted from the raw log and mapped to this field.
network.tls.curve	network.tls.curve	The key exchange curve is extracted from the raw log and mapped to this field.
pid	principal.process.pid, target.process.pid	The process ID (pid) is mapped to either principal.process.pid or target.process.pid depending on the context.
ppid	principal.process.parent_process.pid, target.process.parent_process.pid	The parent process ID (ppid) is mapped to either principal.process.parent_process.pid or target.process.parent_process.pid depending on the context.
principal.asset.hostname	principal.asset.hostname	Copied from principal.hostname.
principal.asset.ip	principal.asset.ip	Copied from principal.ip.
principal.platform	principal.platform	Set by the parser logic based on the operating system (e.g., LINUX).
principal.port	principal.port	The port number associated with the principal.
principal.user.group_identifiers	principal.user.group_identifiers	Group IDs associated with the principal user.
process.name	target.process.file.full_path
receiveTimestamp	(Not Mapped)	This field is the log receive timestamp and is not mapped to the IDM object in the UDM.
res	security_result.action_details	The result (res) is mapped to security_result.action_details.
_Resource_Id	target.resource.product_object_id
resource.labels	(Not Mapped)	These labels are present in some raw logs but are not mapped to the IDM object in the UDM.
resource.type	(Not Mapped)	This field is present in some raw logs but is not mapped to the IDM object in the UDM.
security_result.action	security_result.action	Set by the parser logic based on the res field (e.g., ALLOW, BLOCK).
security_result.detection_fields	security_result.detection_fields	Various fields from the raw log are added as key-value pairs to this array for context.
security_result.rule_id	security_result.rule_id	Set by the parser logic, often to the type_name for syscall events.
security_result.severity	security_result.severity	Set by the parser logic based on the severity level in the raw log.
security_result.summary	security_result.summary	A summary of the event, often derived from the msg field or other relevant fields.
ses	network.session_id	The session ID (ses) is mapped to network.session_id.
source	(Not Mapped)	This field contains metadata about the log source and is not mapped to the IDM object in the UDM.
subj	(Processed)	The subject field (subj) is processed to extract user and security context information.
syscall	security_result.about.labels.Syscall	The syscall number is added as a label within security_result.about.
target.administrative_domain	target.administrative_domain	The domain of the target user.
target.group.group_display_name	target.group.group_display_name	The name of the target group.
target.ip	target.ip	The IP address of the target.
target.port	target.port	The port number associated with the target.
target.process.command_line	target.process.command_line	The command line of the target process.
target.resource.type	target.resource.type	The type of the target resource, set by the parser logic (e.g., CREDENTIAL, SETTING).
target.user.attribute.permissions	target.user.attribute.permissions	Permissions related to the target user.
target.user.group_identifiers	target.user.group_identifiers	Group IDs associated with the target user.
target.user.userid	target.user.userid	The user ID of the target.
TenantId	metadata.product_deployment_id
textPayload	(Processed)	The text payload of the log, processed by the parser to extract various fields.
timestamp	metadata.event_timestamp	The timestamp of the event.
tty	security_result.about.labels.tty	The tty is added as a label within security_result.about.
type	metadata.product_event_type	The event type (type) is mapped to metadata.product_event_type.
uid	target.user.userid	The user ID (uid) is mapped to target.user.userid.

UDM mapping delta reference

On September 23, 2025, Google SecOps released a new version of the Okta parser, which includes significant changes to the mapping of Okta log fields to UDM fields and changes to the mapping of event types.

Log-field mapping delta

The following table lists the mapping delta for Okta log-to-UDM fields exposed prior to September 23, 2025 and subsequently (listed in the Old mapping and Current mapping columns, respectively).

Log field	Old mapping	Current mapping	Reference log sample
1.1.1.1 (ip address)	src.ip	principal.ip	"<163>Apr 10 09:00:05 hostname.com sshd[3318513]: Accepted password for abc from 1.1.1.1 port 33988 ssh2"
1.1.1.1 (ip address)	principal.ip	target.ip	"<29>Oct 5 08:37:16 abc ProxySG: E0000 Access Log HTTP (main): Connecting to server 1.1.1.1 on port 4433.(0) NORMAL_EVENT alog_stream_http.cpp 261"
abc (user)	principal.user.userid	target.user.userid	"<85>Feb 27 08:26:55 offozcav login: FAILED LOGIN 1 FROM ::ffff:1.1.1.1 FOR abc, Authentication failure\r\n\r\n"
abc.abc (user)	principal.user.userid	target.user.userid	"<86>Feb 27 08:29:19 offozcav login: LOGIN ON pts/43 BY abc.abc FROM\r\n\r\n::ffff:1.1.1.1"
COMMAND	principal.process.command_line	target.process.command_line	"<85>Sep 24 14:33:59 abc sudo: abc : \r\nTTY=unknown ; PWD=/abc ; USER=abc ; COMMAND=/sbin/iptables -t nat -nL \r\n--line-number"
exe	target.process.file.full_path	principal.process.file.full_path
_ItemId	additional.fields	metadata.product_log_id
metadata.product_event_type	PATH	SECCOMP
process.name	principal.process.file.full_path	target.process.file.full_path
_ResourceId	additional.fields	target.resource.product_object_id
TenantId	additional.fields	metadata.product_deployment_id
uid	principal.user.userid	target.user.userid
USER	principal.user.user_display_name	target.user.userid	"<85>Sep 24 14:33:59 abc sudo: abc : \r\nTTY=unknown ; PWD=/abc ; USER=abc ; COMMAND=/sbin/iptables -t nat -nL \r\n--line-number"
user	principal.user.userid	target.user.userid	"29>Jan 16 11:28:00 san-auth-1-irl2 tac_plus[17329]: login failure: user 1.1.1.1 (1.1.1.1) vty0"
user	principal.user.userid	target.user.userid	"<87>Jul 15 10:27:01 xpgjrconfdb01 crond[1045]: pam_unix(crond:account): expired password for user root (password aged)"

Event-type mapping delta

Multiple events that were classified before as generic event are now properly classified with meaningful event types.

The following table lists the delta for the handling of Okta event types prior to September 23, 2025 and subsequently (listed in the Old event_type and Current event-type columns respectively).

eventType from log	Old event_type	Current event_type
aix_event_type=CRON_Start	USER_LOGIN	PROCESS_LAUNCH
CRYPTO_KEY_USER	NETWORK_CONNECTION	USER_LOGIN
FILE_Mknod	USER_LOGIN	FILE_CREATION
FILE_Rename	USER_LOGIN	FILE_MODIFICATION
FILE_Stat	USER_LOGIN	FILE_OPEN
FILE_Unlink	USER_LOGIN	FILE_DELETION
FS_Chabc	USER_LOGIN	PROCESS_UNCATEGORIZED
FS_Mkdir	USER_LOGIN	FILE_CREATION
FS_Rmdir	USER_LOGIN	FILE_DELETION
PROC_Execute	USER_LOGIN	PROCESS_LAUNCH
type=ANOM_ABEND	STATUS_UPDATE	PROCESS_TERMINATION
type=ANOM_PROMISCUOUS		SETTING_MODIFICATION
type=CRED_REFR	USER_LOGIN	USER_CHANGE_PERMISSIONS
type=PROCTILE	PROCESS_UNCATEGORIZED	PROCESS_LAUNCH
type=SERVICE_START	USER_RESOURCE_ACCESS	SERVICE_START
type=SERVICE_STOP	USER_RESOURCE_ACCESS	SERVICE_STOP
type=USER_ACCT	USER_LOGIN/SETTING_MODIFICTION	USER_LOGIN
type=USER_MGMT	SETTING_MODIFICATION/GROUP_MODIFICATION	GROUP_MODIFICATION
USER_ERR	USER_LOGOUT	USER_LOGIN

Additional changes

• Removed duplicate mapping of res from security_result.description. It's captured in security_result.action_details.
• Removed unnecessary auditd_msg_data from additional fields.
• Removed unnecessary auditd_msg_data from security_result.summary.
• When type=ADD_USER, removed duplicate mapping of acct to target.user.display_name. It's already mapped under target.user.userid.
• Removed duplicate mapping of comm from principal.process.command_line and principal.process.file.names. It's captured in principal.application.
• Removed duplicate mapping of target.hostname when the value is there in principal.
• Removed unnecessary hard-coded mapping of target.resource.type to SETTING.
• Removed about labels mappings since it is deprecated.
• Corrected mapping: IP's now route to principal.ip, not principal.hostname.
• Fixed the repetition of events being generated.

Need more help? Get answers from Community members and Google SecOps professionals.