# Search raw logs

Supported in:

When you perform a search, Google Security Operations first examines the security data that's been ingested, parsed, and normalized. If the information isn't found in the normalized data, you can use **Raw Log Search** to examine the raw, unparsed logs. You can also use regular expressions to examine the raw logs in more detail.

> [!NOTE]
> **Note:** The search field is limited to 150 characters.

You can use **Raw Log Search** to investigate artifacts that appear in logs (but aren't indexed), including:

- Usernames
- Filenames
- Registry keys
- Command-line arguments
- Raw HTTP request-related data
- Domain names based on regular expressions
- [Asset namespaces](https://docs.cloud.google.com/chronicle/docs/investigation/asset-namespaces) and addresses

## Primary use cases for raw log search

Raw log search is a specialized investigation tool designed for the following
use cases:

- **Verify normalization and troubleshoot parsers**: Compare raw log entries with generated UDM events to identify normalization gaps, verify field mapping, or pinpoint unparsed logs.
- **Deep forensic investigation and threat hunting**: Search for uncommon or unindexed indicators of compromise (IOCs), command-line arguments, registry keys, file paths, or specific strings that aren't mapped to structured UDM fields.
- **Investigate unstructured and custom data**: Query newly onboarded or proprietary log sources immediately without waiting for a UDM parser to be developed.

> [!NOTE]
> **Note:** Raw log search is designed as an interactive investigation and threat-hunting tool for these specific workflows. It isn't designed as an ingestion accounting or volumetric audit ledger. To validate ingestion completeness or volumetric data parity, use [UDM Search](https://docs.cloud.google.com/chronicle/docs/investigation/udm-search), [Statistics and aggregations in UDM Search](https://docs.cloud.google.com/chronicle/docs/investigation/statistics-aggregations-in-udm-search), or [ingestion health metrics](https://docs.cloud.google.com/chronicle/docs/ingestion/ingestion-notifications-for-health-metrics).

## Raw log search

You can perform a raw log search using the **Search** bar, located on the landing page or
in the menu bar. Choose one of the following methods:

### Use the `raw=` format

You can query raw logs by using the `raw=` format. This is the recommended method.

- To search for a substring, enclose the search term in quotes. For example, `raw = "ABC"`.
- To search using a regular expression, enclose the expression in forward slashes (/). For example, `raw = /AB*C/`.
- Optional: Filter by parsed status using the `parsed` keyword:
  - `parsed = false`: Displays unparsed logs where no UDM event was generated.
  - `parsed = true`: Displays logs that were successfully parsed into UDM events.

### Legacy method: Use the Raw Log Search prompt

- In the **Search** bar, enter a search string with at least four characters (for example, an MD5 hash), including wildcards.
- If the search returns no results, the **Raw Log Search** option appears.
- Optional: Specify the **Start Time** and **End Time**. The default range is the last 7 days.
- Optional: In the **Log Sources** list, select one or more log sources. The default setting is **All**.
- Click **Search**.

Events associated with the search string are displayed in the **Timeline** . Click an event (or the raw log icon)
to open the corresponding raw log.
The search query along with the applied filters are also visible
on the **Raw Log Search** page.

> [!IMPORTANT]
> **Important:** Due to internal constraints in the legacy raw log search system, the corresponding UDM event might occasionally fail to display next to the raw log entry. This isn't a parsing error but a display limitation of the legacy raw log search system.

## Optimize raw log queries

Raw log searches scan unindexed log payloads and are typically slower than UDM
searches. To improve your search performance, limit the amount of data you
conduct your query over by changing the search settings:

- Time range selector: Limit the time range of the data over which you run your query.
- Log Sources selector: Limit your raw log search to only the logs from specific sources, as opposed to all of your log sources. From the **Log
  Sources** menu, select one or more log sources (the default is **All** ), or append the `log_source` filter directly to your query (`log_source = IN["<var>LOG_SOURCE_NAME</var>"]`).
- Regular expressions: Use a specific regular expression. For example, `raw = /goo\w{3}\.com/` matches against `google.com`, `goodle.com`, and `goog1e.com` to further limit the scope of your raw log search.

### Targeted raw log search examples

Combine the `raw =` prefix with `log_source` and `parsed` filters to restrict
your search to relevant feeds and prevent high-volume log sources from
truncating your results:

- **Search a single log source for an exact string**:

      raw = "<var>SEARCH_TERM</var>" and log_source = IN["<var>LOG_SOURCE_NAME</var>"]

- **Search a subset of log sources**:

      raw = "<var>SEARCH_TERM</var>" and log_source = IN["<var>LOG_SOURCE_1</var>", "<var>LOG_SOURCE_2</var>"]

- **Search unparsed logs with a regular expression**:

      raw = /<var>REGULAR_EXPRESSION</var>/ and parsed = false and log_source = IN["<var>LOG_SOURCE_NAME</var>"]

## Trend over time

Use the **Trend Over Time** graph to understand the distribution of raw logs over the time of
your search. You can apply filters on the graph to look for parsed logs and
raw logs. Click **Expand more** (or **Expand less**) to expand or collapse the graph.

## Raw log results

When you run a raw log search, the results are a combination of UDM events and
entities generated by the raw logs that match your searches, along with the
raw logs. You can explore the search results further by clicking any of the
results:

- UDM event or entity: If you click a UDM event or entity, Google SecOps
  shows any related events and entities, along with the raw log associated
  with that item.

- Raw log: If you click a raw log, Google SecOps shows
  you the entire raw log line, along with the source for that log.

### Result limits and sampling

Raw log searches enforce the following result and query limits:

- **Large-batch result limit**: For recently ingested logs stored in batched storage before export to long-term columnar storage, there is a limit of 1,000 results returned from large log batches containing 1,000 or more events. Once logs are exported to long-term columnar storage, this 1,000-result batch limit doesn't apply.
- **Total UI and CSV export limit**: A single raw log search in the Google SecOps console or CSV export returns up to 10,000 total results across all queried log sources. If a query matches more than 10,000 entries, Google SecOps returns a representative sample of 10,000 results.
- **API result limit** : The Raw Log Search API (`SearchRawLogs`) enforces a maximum limit of 10,000 entries per query request (`page_size`).
- **Search query length**: The raw log search input field is limited to 150 characters.

When you search across multiple log sources simultaneously (leaving the **Log
Sources** menu set to **All**), Google SecOps executes queries in
parallel across distributed storage shards with protective query limits applied
before in-memory filtering to maintain fast response times. Because of this
distributed architecture, total returned result counts can vary between broad
multi-source queries and single-source queries due to the following factors:

- **Varying ingestion volumes**: High-volume log sources generate large batches that reach the 1,000-result batch limit faster than lower-volume sources.
- **Distributed batch sampling**: Large batches containing 1,000 or more events are limited to 1,000 returned results before the overall 10,000-result query sampling limit is applied, which changes the proportion of sampled logs returned across queries.
- **Recent and historical storage tiers**: Recently ingested logs are queried from low-latency batched storage for immediate search availability, where the 1,000-result limit on large batches applies. Once logs are exported to long-term columnar storage, queries scan individual log entries directly without the 1,000-result batch limit (up to the 10,000-result total query limit), providing more consistent result counts for historical searches.

> [!NOTE]
> **Note:** Result count variations across multi-source queries are by design to maintain query responsiveness across large datasets and don't indicate data loss or an ingestion failure. To retrieve a complete, unsampled result set, filter your query to specific log sources and narrow the search time range to avoid hitting batch and overall query sampling limits.

## Download raw log results

To download raw log results to a CSV file, in the **Raw Logs** results table,
click **More** \> **Download as CSV**.

By default, the CSV file always includes the **Raw Log** column. You can use the **Columns** button to open the **Column Manager** and remove or add the **Timestamp** and **Type** columns. If you add any other columns, you'll see them in the table, but they won't be downloaded in the CSV file.

To download UDM fields, run a UDM search, select the required columns, and then download as a CSV file.

If the `Failed to execute 'showSaveFilePicker'` error appears in Google Chrome,
perform the following steps:

1. Go to **Devices** \> **Chrome** \> **Settings**.

2. Select **Users \& browsers**.

3. Locate the **File system write access** setting.

4. Add `prefix.backstory.chronicle.security` to the list of allowed sites.

## Regular expressions

You can use regular expressions in Google SecOps to search for and match sets of character
strings in your security data. Regular expressions
help to narrow your search using fragments of information rather than requiring an exact match.

To run a search using regular expression syntax:

- In **SIEM Search** , enter `raw = /YOUR_REGEX/` in the search field and click **Run Search**.
- In the legacy **Search** bar:
  1. Enter a regular expression (4 to 66 characters long) in the **Search** field.
  2. Select the **Run Query as Regex** checkbox and click **Search** (or **Update Search** on the **Raw Log Search** results page).

The Google SecOps regular expression infrastructure is based on [Google RE2](https://github.com/google/re2/wiki/Syntax), an open-source regular expression engine. Google SecOps uses the same regular expression syntax.

> [!NOTE]
> **Note:** For predictable performance, the RE2 engine intentionally omits certain advanced features, such as lookahead and lookbehind assertions. For more information, see [RE2 syntax](https://github.com/google/re2/wiki/Syntax).

The following table highlights some of the common regular expression syntaxes you can use for your searches.

|---|---|
| Any character | . |
| x number of any characters | {x} |
| Character class | \[xyz\] |
| Negated character class | \[\^xyz\] |
| Alphanumeric (0-9A-Za-z) | \[\[:alnum:\]\] |
| Alphabetic (A-Za-z) | \[\[:alpha:\]\] |
| Digits (0-9) | \[\[:digit:\]\] |
| Lower case (a-z) | \[\[:lower:\]\] |
| Upper case (A-Z) | \[\[:upper:\]\] |
| Word characters (0-9A-Za-z_) | \[\[:word:\]\] |
| Hex digit (0-9A-Fa-f) | \[\[:xdigit:\]\] |
| Question mark symbol (?) | Matches zero or one occurrence of the preceding element. |
| Asterisk (\*) | Matches zero or more occurrences of the preceding character or group. |
| Plus sign (+) | Matches one or more occurrences of the preceding character or group. |

<br />

The following examples illustrate how you can use regular expressions to search data:

- `goo.le\.com`---Matches any string that starts with `goo`, followed by any single character, followed by `le.com`, such as `google.com` or `goo0le.com`.
- `goo\w{3}\.com`---Matches strings that start with `goo`, followed by exactly three word characters (`\w`), and ending with `.com`. Examples include `google.com`, `goojle.com`, or `goodle.com`.
- `[[:digit:]]\.[[:alpha:]]`---matches a string that has a single digit, followed by a dot (.), followed by a single alphabetic character, such as `34323.system`, `23458.office`, or `897.net`.

### Sample regular expressions to search for Windows logs

This section provides regular expression query strings you can use with Google SecOps **Raw Log Search** to find commonly monitored Windows events. These examples assume the Windows log messages are in JSON format.

For more information about commonly monitored Windows Event IDs, see [Events to Monitor](https://docs.microsoft.com/en-us/windows-server/identity/ad-ds/plan/appendix-l--events-to-monitor). The examples provided follow a similar pattern, described in these use cases.

|---|---|
| **Use Case: Return events with the EventID 1150** ||
| Regular expression string: | `\"EventID\"\:\s*1150` |
| Values Matched: | `"EventID":1150` |

|---|---|
| **Use Case:Return events with an Event ID that is either 1150 or 1151** ||
| Regular expression string | `(?:\"EventID\"\:\s*)(?:1150|1151)` |
| Values Matched | `"EventID":1150` and `"EventID":1151` |

|---|---|
| **Use Case: Return events with an Event ID that is either 1150 or 1151, and with ThreadID 9092** ||
| Regular expression string | `(?:\"EventID\"\:\s*)(?:1150|1151).*(?:\"ThreadID\"\:\s*9092)` |
| Values Matched | `"EventID":1150 <...any number of characters...> "ThreadID":9092` and `"EventID":1151 <...any number of characters...> "ThreadID":9092` |

#### Find account management events

These regular expression query strings identify common account management events using the EventID attribute.

|---|---|
| **Type of Event** | **Regular Expression** |
| User Account Created | `"EventID\"\:\s*4720` |
| User Account Enabled | `"EventID\"\:\s*4722` |
| User Account Disabled | `"EventID\"\:\s*4725` |
| User Account Deleted | `"EventID\"\:\s*4726` |
| User Rights Modification | `"EventID\"\:\s*4703` |
| Member Added to Security Enabled Global Group | `"EventID\"\:\s*4728` |
| Member Removed from Security Enabled Global Group | `"EventID\"\:\s*4729` |
| Security Enabled Global Group was Deleted | `"EventID\"\:\s*4730` |

#### Find logon success events

These regular expression query strings identify types of successful logon events using the EventID and LogonType attributes.

|---|---|
| **Type of Event** | **Regular Expression** |
| Logon Success | `"EventID\"\:\s*4624` |
| Logon Success - Interactive (LogonType=2) | `"EventID\"\:\s*4624.*?LogonType\"\:\s*\"2\"` |
| Logon Success - Batch Login (LogonType=4) | `"EventID\"\:\s*4624.*?LogonType\"\:\s*\"4\"` |
| Logon Success - Service Login (LogonType=5) | `"EventID\"\:\s*4624.*?LogonType\"\:\s*\"5\"` |
| Logon Success - RemoteInteractive Login (LogonType=10) | `"EventID\"\:\s*4624.*?LogonType\"\:\s*\"10\"` |
| Logon Success - Interactive, Batch, Service, or RemoteInteractive | `(?:"EventID\"\:\s*4624.*?LogonType\"\:\s*\")(?:2|4|5|10)\"` |

#### Find logon failure events

These regular expression query strings identify types of failed logon events using the EventID and LogonType attributes.

|---|---|
| **Type of Event** | **Regular Expression** |
| Logon Failure | `"EventID\"\:\s*4625` |
| Logon Failure - Interactive (LogonType=2) | `"EventID\"\:\s*4625.*?LogonType\"\:\s*\"2\"` |
| Logon Failure - Batch Login (LogonType=4) | `"EventID\"\:\s*4625.*?LogonType\"\:\s*\"4\"` |
| Logon Failure - Service Login (LogonType=5) | `"EventID\"\:\s*4625.*?LogonType\"\:\s*\"5\"` |
| Logon Failure - RemoteInteractive Login (LogonType=10) | `"EventID\"\:\s*4625.*?LogonType\"\:\s*\"10\"` |
| Logon Failure - Interactive, Batch, Service, or RemoteInteractive | `(?:"EventID\"\:\s*4625.*LogonType\"\:\s*\")(?:2|4|5|10)\"` |

#### Find process, service and task events

These regular expression query strings identify certain process and service events using the EventID attribute.

|---|---|
| **Type of Event** | **Regular Expression** |
| Process Start | `"EventID\"\:\s*4688` |
| Process Exit | `"EventID\"\:\s*4689` |
| Service Installed | `"EventID\"\:\s*4697` |
| New Service Created | `"EventID\"\:\s*7045` |
| Schedule Task Created | `"EventID\"\:\s*4698` |

#### Find events related to object access

These regular expression query strings identify different types of process and service related events using the EventID attribute.

|---|---|
| **Type of Event** | **Regular Expression** |
| Audit Log Cleared | `"EventID\"\:\s*1102` |
| Object Access Attempted | `"EventID\"\:\s*4663` |
| Share Accessed | `"EventID\"\:\s*5140` |

## Troubleshooting

Use the following table to resolve common raw log search issues:

| Issue | Description | Fix |
|---|---|---|
| Inconsistent result counts across searches | Querying across multiple log sources (**All**) returns varying total result counts compared to single-source queries for recently ingested logs due to the 1,000-result limit on large batches (batches with 1,000 or more events) in batched storage and distributed sampling. | Select specific log sources from the **Log Sources** menu or filter using `log_source = IN["LOG_SOURCE_NAME"]`, and narrow the search time range. |
| Truncated or sampled results (10,000-result limit reached) | The search query matched more than 10,000 total logs across the UI, CSV export, or `SearchRawLogs` API (`page_size`), returning only a 10,000-result sample. | Shorten the query time window (for example, to 1 hour) and add specific RE2 regular expressions or `parsed = false` filters. |
| Volumetric ingestion audit discrepancies | Raw log search result counts don't match total ingested log volumes because protective batch and shard limits are applied before in-memory filtering. | Use [UDM Search](https://docs.cloud.google.com/chronicle/docs/investigation/udm-search), [Statistics and aggregations in UDM Search](https://docs.cloud.google.com/chronicle/docs/investigation/statistics-aggregations-in-udm-search), or [ingestion health metrics](https://docs.cloud.google.com/chronicle/docs/ingestion/ingestion-notifications-for-health-metrics) to verify ingestion completeness and volumetric parity. |
| Query exceeds character limit | The search string exceeds the 150-character limit for raw log queries. | Simplify the regular expression or split the investigation into smaller, separate queries targeting individual log sources. |

**Need more help?** [Get answers from Community members and Google SecOps professionals.](https://security.googlecloudcommunity.com/)