Change log for CLEARPASS

Date Changes
2025-10-29 Enhancement:
- `event.idm.read_only_udm.principal.user.userid`: Removed mapping of `vd` from `event.idm.read_only_udm.principal.user.userid` UDM field since it is a virtual domain not user.
- `event.idm.read_only_udm.principal.user.userid`: Mapped `user` raw log field to `event.idm.read_only_udm.principal.user.userid` UDM field.
- `event.idm.read_only_udm.observer.resource.attribute.labels`: Newly mapped `vd` raw log field(s) with `event.idm.read_only_udm.observer.resource.attribute.labels` UDM field.
- `event.idm.read_only_udm.network.application_protocol`: Newly mapped `ui` raw log field(s) with `event.idm.read_only_udm.network.application_protocol` UDM field.
- `event.idm.read_only_udm.principal.ip`, `event.idm.read_only_udm.principal.asset.ip`: Newly mapped `ui` raw log field(s) with `event.idm.read_only_udm.principal.ip`, `event.idm.read_only_udm.principal.asset.ip` UDM field(s).
- Renamed from _observer to event.idm.read_only_udm.observer.
- ui: Transformed using grok pattern (?\\w+)\((?%{IP})\) to extract proto and ui_ip.
- Added conditional check for level: if the value is alert, maps HIGH to event.idm.read_only_udm.security_result.severity.
2025-10-01 Enhancement:
- Added a new grok pattern to parse the new format of SYSLOG + KV logs.
- event.idm.read_only_udm.principal.ip: Newly mapped "CppmNode.CPPM-Node" raw log field with "event.idm.read_only_udm.principal.ip" UDM field.
- event.idm.read_only_udm.principal.asset.ip: Newly mapped "CppmNode.CPPM-Node" raw log field with "event.idm.read_only_udm.principal.asset.ip" UDM field.
- event.idm.read_only_udm.principal.user.group_identifiers: Newly mapped "Auth.Roles" raw log field with "event.idm.read_only_udm.principal.user.group_identifiers" UDM field.
- KVData: Changed the key-value pair delimiter from "|" to ";" to correctly parse raw log values that contain a "|" character, such as "Auth.Enforcement-Profiles".
2025-09-25 Enhancement:
- Added a new grok pattern to parse the new format of SYSLOG.
- event.idm.read_only_udm.additional.fields: Newly mapped `ExporterName` raw log field with `event.idm.read_only_udm.additional.fields` UDM field.
- event.idm.read_only_udm.security_result.detection_fields: Newly mapped `Category` raw log field with `event.idm.read_only_udm.security_result.detection_fields` UDM field.
- event.idm.read_only_udm.security_result.action: Newly mapped `Action` raw log field with `event.idm.read_only_udm.security_result.action` UDM field.
- event.idm.read_only_udm.security_result.action_details: Newly mapped `Action` raw log field with `event.idm.read_only_udm.security_result.action_details` UDM field.
- event.idm.read_only_udm.principal.ip: Newly mapped `Remote_IP_Address` raw log field with `event.idm.read_only_udm.principal.ip` UDM field.
- event.idm.read_only_udm.principal.asset.ip: Newly mapped `Remote_IP_Address` raw log field with `event.idm.read_only_udm.principal.asset.ip` UDM field.
- event.idm.read_only_udm.principal.port: Newly mapped `port` raw log field with `event.idm.read_only_udm.principal.port` UDM field.
- event.idm.read_only_udm.intermediary.ip: Newly mapped `InterIP` raw log field with `event.idm.read_only_udm.intermediary.ip` UDM field.
- event.idm.read_only_udm.intermediary.asset.ip: Newly mapped `InterIP` raw log field with `event.idm.read_only_udm.intermediary.asset.ip` UDM field.
- event.idm.read_only_udm.security_result.description: Newly mapped `sdescription` raw log field with `event.idm.read_only_udm.security_result.description` UDM field.
- event.idm.read_only_udm.security_result.detection_fields: Newly mapped `code` raw log field with `event.idm.read_only_udm.security_result.detection_fields` UDM field.
- event.idm.read_only_udm.principal.ip: Newly mapped `ip` raw log field with `event.idm.read_only_udm.principal.ip` UDM field.
- event.idm.read_only_udm.principal.asset.ip: Newly mapped `ip` raw log field with `event.idm.read_only_udm.principal.asset.ip` UDM field.
- Removed the drop condition where Description is having `No new updates available.` and parsed the log properly into respective UDM fields.
2025-09-16 Enhancement:
- `event.idm.read_only_udm.principal.application`: Newly mapped `server` raw log field with `event.idm.read_only_udm_principal.application` UDM field.
- Added Grok patterns to parse new log formats.
2025-09-05 Enhancement:
- Added new grok patterns to parse new log events.
2025-08-28 Enhancement:
- event.idm.read_only_udm.metadata.product_log_id: Newly mapped logid raw log field with event.idm.read_only_udm.metadata.product_log_id UDM field.
- event.idm.read_only_udm.metadata.description: Newly mapped logdesc, Notice_Type raw log field(s) with event.idm.read_only_udm.metadata.description UDM field.
- event.idm.read_only_udm.metadata.event_timestamp: Newly mapped from a combination of date, time, and tz raw log field(s) with event.idm.read_only_udm.metadata.event_timestamp UDM field.
- event.idm.read_only_udm.metadata.event_timestamp: Newly mapped from a combination of event_time, event_date raw log field(s) with event.idm.read_only_udm.metadata.event_timestamp UDM field.
- event.idm.read_only_udm.metadata.product_event_type: Newly mapped Event_ID raw log field with event.idm.read_only_udm.metadata.product_event_type UDM field.
- event.idm.read_only_udm.principal.hostname: Newly mapped devname raw log field with event.idm.read_only_udm.principal.hostname UDM field.
- event.idm.read_only_udm.principal.asset.hostname: Newly mapped devname raw log field with event.idm.read_only_udm.principal.asset.hostname UDM field.
- event.idm.read_only_udm.principal.asset.asset_id: Newly mapped devid raw log field with event.idm.read_only_udm.principal.asset.asset_id UDM field.
- event.idm.read_only_udm.principal.user.userid: Newly mapped vd, User_Name raw log field(s) with event.idm.read_only_udm.principal.user.userid UDM field.
- event.idm.read_only_udm.principal.ip: Newly mapped Remote_IP_Address raw log field with event.idm.read_only_udm.principal.ip UDM field.
- event.idm.read_only_udm.principal.asset.ip: Newly mapped Remote_IP_Address raw log field with event.idm.read_only_udm.principal.asset.ip UDM field.
- event.idm.read_only_udm.target.hostname: Newly mapped hostname raw log field with event.idm.read_only_udm.target.hostname UDM field.
- event.idm.read_only_udm.target.asset.hostname: Newly mapped hostname raw log field with event.idm.read_only_udm.target.asset.hostname UDM field.
- event.idm.read_only_udm.target.user.userid: Newly mapped target_user_userid raw log field with event.idm.read_only_udm.target.user.userid UDM field.
- event.idm.read_only_udm.security_result.description: Newly mapped msg raw log field with event.idm.read_only_udm.security_result.description UDM field.
- event.idm.read_only_udm.security_result.severity: Newly mapped level, log_level, severity raw log field(s) with event.idm.read_only_udm.security_result.severity UDM field.
- event.idm.read_only_udm.security_result.severity_details: Newly mapped level raw log field with event.idm.read_only_udm.security_result.severity_details UDM field.
- event.idm.read_only_udm.security_result.action_details: Newly mapped action_details raw log field with event.idm.read_only_udm.security_result.action_details UDM field.
- event.idm.read_only_udm.additional.fields: Newly mapped log_time, type, status,subtype, eventtime, Config_Method, module, interface, event_label, event_code, trap_name, snmp_oid, log_source raw log field(s) to corresponding keys in event.idm.read_only_udm.additional.fields.
- Renamed from Device-Name to Device_Name.
- Renamed from Config-Method to Config_Method.
- Renamed from Event-ID to Event_ID.
- Renamed from Notice-Type to Notice_Type.
- Renamed from User-Name to User_Name.
- Renamed from Remote-IP-Address to Remote_IP_Address.
- The condition for mapping event.idm.read_only_udm.network.application_protocol was updated to include "SSH".
- The condition for setting event.idm.read_only_udm.security_result.severity to "MEDIUM" was updated to include a check for [Level] == "notice".
2025-08-20 Enhancement:
- Added a new grok pattern for `not_json` data field to parse the following drop and unparsed logs.
- `event.idm.read_only_udm.metadata.product_version` : Newly mapped `swVersion` raw log field with `event.idm.read_only_udm.metadata.product_version` UDM field.
- `event.idm.read_only_udm.additional.fields` : Newly mapped `enterpriseId` and `software` raw log fields with `event.idm.read_only_udm.additional.fields` UDM field.
- `event.idm.read_only_udm.principal.ip` : Newly mapped `ip` raw log fields with `event.idm.read_only_udm.principal.ip` and `event.idm.read_only_udm.principal.asset.ip` UDM fields.
- `event.idm.read_only_udm.principal.ip` : Newly mapped `CppmNode.CPPM-Node` raw log fields with `event.idm.read_only_udm.principal.ip` and `event.idm.read_only_udm.principal.asset.ip` UDM fields if `CppmNode.CPPM-Node` is not equal to `ip` else mapped `CppmNode.CPPM-Node` raw log fields with `event.idm.read_only_udm.additional.fields` UDM fields.
- `event.idm.read_only_udm.security_result.action_details` : Newly mapped `Action` raw log field with `event.idm.read_only_udm.security_result.action_details` UDM field.
- Added a new gsub on `_principal_ip` to replace "\\\\" with "".
2025-07-18 Enhancement:
- Added a new grok patterns for `not_json` data field to parse the following drop and unparsed logs.
- Added a new gsub to parse the following drop and unparsed logs.
- `event.idm.read_only_udm.security_result.description` : Newly mapped `messagedetail` raw log field with `event.idm.read_only_udm.security_result.description` UDM field.
- `event.idm.read_only_udm.additional.fields` : Newly mapped `Thread_id`,`servicename` , and `Req_id` raw log fields with `event.idm.read_only_udm.additional.fields` UDM field.
- `event.idm.read_only_udm.security_result.detection_fields` : Newly mapped `Internalserviceid`, `InternalChallengeID`, `Entityid`, `handlervalue`, and `instanceid` raw log fields with `event.idm.read_only_udm.security_result.detection_fields` UDM field.
- `event.idm.read_only_udm.metadata.description` : Newly mapped `threadrequestinfo` raw log field with `event.idm.read_only_udm.metadata.description` UDM field.
- Modified the mapping of `timestamp` raw log field to correctly map with `event.idm.read_only_udm.metadata.event_timestamp` UDM field.
2025-07-14 Enhancement:
- Added a gsub to replace "\\r" with "".
- Added a Grok pattern to parse the raw log fields.
- Removed a gsub to replace "\r\n" with " ".
- event.idm.read_only_udm.principal.process.pid: Newly mapped `prin_pid` field with `event.idm.read_only_udm.principal.process.pid`.
- event.idm.read_only_udm.additional.fields: Newly mapped `host_host` field with `event.idm.read_only_udm.additional.fields`.
- event.idm.read_only_udm.security_result.detection_fields: Newly mapped `id1`,`TACACS_Privilege_Level`,`common_login_status` field with `event.idm.read_only_udm.security_result.detection_fields`.
- Added a grok pattern on "kv_data_2" field to extract "common_request_timestamp".
- event.idm.read_only_udm.metadata.event_timestamp: Newly mapped `common_request_timestamp` field with `event.idm.read_only_udm.metadata.event_timestamp`.
- event.idm.read_only_udm.principal.user.userid: Newly mapped `Common_Username` raw log field with `event.idm.read_only_udm.principal.user.userid`.
- Added a grok pattern on "TACACS_Remote_Address" to extract valid ip format.
- `event.idm.read_only_udm.target.ip` and `event.idm.read_only_udm.target.asset.ip`: Newly mapped `TACACS_Remote_Address` field with `event.idm.read_only_udm.target.ip` and `event.idm.read_only_udm.target.asset.ip`.
2025-06-25 Enhancement:
- event.idm.read_only_udm.metadata.description: Newly mapped `RADIUS.Acct-Service-Name` raw log field with `event.idm.read_only_udm.metadata.description`.
- event.idm.read_only_udm.intermediary.ip: Newly mapped `RADIUS.Acct-NAS-IP-Address` raw log field with `event.idm.read_only_udm.intermediary.ip`.
- event.idm.read_only_udm.principal.labels: Newly mapped `RADIUS.Acct-NAS-Port-Type` raw log field with `event.idm.read_only_udm.principal.labels`.
- event.idm.read_only_udm.network.session_id: Newly mapped `RADIUS.Acct-Session-Id` raw log field with `event.idm.read_only_udm.network.session_id`.
- event.idm.read_only_udm.network.session_duration.seconds: Newly mapped `RADIUS.Acct-Session-Time` raw log field with `event.idm.read_only_udm.network.session_duration.seconds`.
- event.idm.read_only_udm.network.sent_bytes: Newly mapped `RADIUS.Acct-Input-Pkts` raw log field with `event.idm.read_only_udm.network.sent_bytes`.
- event.idm.read_only_udm.network.received_bytes: Newly mapped `RADIUS.Acct-Output-Pkts` raw log field with `event.idm.read_only_udm.network.received_bytes`.
- Added Support to the event_type when `acct_service_name` has "Login" then event_type is "USER_LOGIN", when `acct_service_name` has "Logout" then event_type is "USER_LOGOUT".
2025-05-09 Enhancement:
- Added support to parse new format of SYSLOG + KV logs.
- `event.idm.read_only_udm.metadata.product_version`: Newly mapped `swVersion` raw log field with `event.idm.read_only_udm.metadata.product_version`.
- `event.idm.read_only_udm.additional.fields`: Newly mapped `software` , `code_error` and `enterpriseId` raw log fields with `event.idm.read_only_udm.additional.fields`.
- `event.idm.read_only_udm.metadata.product_log_id`: Newly mapped `eventId` raw log field with `event.idm.read_only_udm.metadata.product_log_id`.
- `event.idm.read_only_udm.principal.mac`: Newly mapped `Common.Host-MAC-Address` raw log field with `event.idm.read_only_udm.principal.mac`.
- `event.idm.read_only_udm.security_result.detection_fields`: Newly mapped `Common.Service` , `Common.Enforcement-Profiles` , `req_time` raw log field with `event.idm.read_only_udm.security_result.detection_fields`.
- `event.idm.read_only_udm.principal.ip` and `event.idm.read_only_udm.principal.asset.ip`: Newly mapped `ip` raw log field with `event.idm.read_only_udm.principal.ip` and `event.idm.read_only_udm.principal.asset.ip`.
- `event.idm.read_only_udm.target.ip` and `event.idm.read_only_udm.target.asset.ip`: Newly mapped `Common.NAS-IP-Address` and `CppmNode.CPPM-Node` raw log field with `event.idm.read_only_udm.target.ip` and `event.idm.read_only_udm.target.asset.ip`.
- `event.idm.read_only_udm.principal.hostname` and `event.idm.read_only_udm.principal.asset.hostname`: Newly mapped `RADIUS.Auth-Source` raw log field with `event.idm.read_only_udm.principal.hostname` and `event.idm.read_only_udm.principal.asset.hostname`.
- `event.idm.read_only_udm.principal.user.group_identifiers`: Newly mapped `roles` raw log field with `event.idm.read_only_udm.principal.user.group_identifiers`.
- `event.idm.read_only_udm.principal.application`: Newly mapped `auth_method` raw log field with `event.idm.read_only_udm.principal.application`.
- `event.idm.read_only_udm.principal.user.userid`: Newly mapped `user_id` raw log field with `event.idm.read_only_udm.principal.user.userid`.
- `event.idm.read_only_udm.security_result.description`: Newly mapped `alerts` raw log field with `event.idm.read_only_udm.security_result.description`.
- If `roles` has value "Authenticated" then map the "event_type" as "USER_LOGIN" else map the "event_type" as "USER_UNCATEGORIZED".
2024-09-12 Enhancement:
- Added support to parse new format of SYSLOG and JSON logs.
2024-08-08 Enhancement:
- Mapped "Acct-NAS-IP-Address" to "principal.ip".
- Mapped "Acct-Username" to "principal.user.userid".
- Mapped "Acct-Calling-Station-Id" to "principal.user.product_object_id".
2024-05-05 Enhancement:
- Handled unparsing SYSLOG format logs.
- Mapped "prin_port" to "principal.port".
- Mapped "agent_ip" to "principal.ip" and "principal.asset.ip".
- Mapped "descr" and "eventDescription" to "metadata.description".
- Mapped "version" to "metadata.product_version".
- Mapped "specificTrap_name", "uptime", "enterprise", "generic_num", "specificTrap_num", and "community" to "additional.fields".
2024-01-11 Enhancement:
- Mapped "Common.NAS-IP-Address" to "target.ip".
- Mapped "Common.Service", "Common.Enforcement-Profiles", and "Common.Login-Status" to "security_result.detection_fields".
2022-08-18 Enhancement:
- Handled the dropped logs which are in CEF format and unparsed logs to improve the parsing rate.
- Mapped "metadata.event_type" to "STATUS_UPDATE" where "principal.hostname/principal.ip" is not null else mapped it as "GENERIC_EVENT".
2022-07-08 Enhancement:
- Modified mapping for "_target_user_groupid" from "target.user.groupid" to "target.user.group_identifiers".
- Modified mapping for "Common.Roles" from "principal.user.groupid" to "principal.user.group_identifiers".