Search raw logs

Supported in:

Google SecOps SIEM

When you perform a search, Google Security Operations first examines the security data that's been ingested, parsed, and normalized. If the information isn't found in the normalized data, you can use Raw Log Search to examine the raw, unparsed logs. You can also use regular expressions to examine the raw logs in more detail.

Note: The search field is limited to 150 characters.

You can use Raw Log Search to investigate artifacts that appear in logs (but aren't indexed), including:

• Usernames
• Filenames
• Registry keys
• Command-line arguments
• Raw HTTP request-related data
• Domain names based on regular expressions
• Asset namespaces and addresses

Primary use cases for raw log search

Raw log search is a specialized investigation tool designed for the following use cases:

• Verify normalization and troubleshoot parsers: Compare raw log entries with generated UDM events to identify normalization gaps, verify field mapping, or pinpoint unparsed logs.
• Deep forensic investigation and threat hunting: Search for uncommon or unindexed indicators of compromise (IOCs), command-line arguments, registry keys, file paths, or specific strings that aren't mapped to structured UDM fields.
• Investigate unstructured and custom data: Query newly onboarded or proprietary log sources immediately without waiting for a UDM parser to be developed.

Note: Raw log search is designed as an interactive investigation and threat-hunting tool for these specific workflows. It isn't designed as an ingestion accounting or volumetric audit ledger. To validate ingestion completeness or volumetric data parity, use UDM Search, Statistics and aggregations in UDM Search, or ingestion health metrics.

Raw log search

You can perform a raw log search using the Search bar, located on the landing page or in the menu bar. Choose one of the following methods:

Use the raw= format

You can query raw logs by using the raw= format. This is the recommended method.

• To search for a substring, enclose the search term in quotes. For example, raw = "ABC".
• To search using a regular expression, enclose the expression in forward slashes (/). For example, raw = /AB*C/.
• Optional: Filter by parsed status using the parsed keyword:
  • parsed = false: Displays unparsed logs where no UDM event was generated.
  • parsed = true: Displays logs that were successfully parsed into UDM events.

Legacy method: Use the Raw Log Search prompt

• In the Search bar, enter a search string with at least four characters (for example, an MD5 hash), including wildcards.
• If the search returns no results, the Raw Log Search option appears.
• Optional: Specify the Start Time and End Time. The default range is the last 7 days.
• Optional: In the Log Sources list, select one or more log sources. The default setting is All.
• Click Search.

Events associated with the search string are displayed in the Timeline. Click an event (or the raw log icon) to open the corresponding raw log. The search query along with the applied filters are also visible on the Raw Log Search page.

Important: Due to internal constraints in the legacy raw log search system, the corresponding UDM event might occasionally fail to display next to the raw log entry. This isn't a parsing error but a display limitation of the legacy raw log search system.

Optimize raw log queries

Raw log searches scan unindexed log payloads and are typically slower than UDM searches. To improve your search performance, limit the amount of data you conduct your query over by changing the search settings:

• Time range selector: Limit the time range of the data over which you run your query.
• Log Sources selector: Limit your raw log search to only the logs from specific sources, as opposed to all of your log sources. From the Log Sources menu, select one or more log sources (the default is All), or append the log_source filter directly to your query (log_source = IN["<var>LOG_SOURCE_NAME</var>"]).
• Regular expressions: Use a specific regular expression. For example, raw = /goo\w{3}\.com/ matches against google.com, goodle.com, and goog1e.com to further limit the scope of your raw log search.

Targeted raw log search examples

Combine the raw = prefix with log_source and parsed filters to restrict your search to relevant feeds and prevent high-volume log sources from truncating your results:

• Search a single log source for an exact string:

  raw = "<var>SEARCH_TERM</var>" and log_source = IN["<var>LOG_SOURCE_NAME</var>"]
  

• Search a subset of log sources:

  raw = "<var>SEARCH_TERM</var>" and log_source = IN["<var>LOG_SOURCE_1</var>", "<var>LOG_SOURCE_2</var>"]
  

• Search unparsed logs with a regular expression:

  raw = /<var>REGULAR_EXPRESSION</var>/ and parsed = false and log_source = IN["<var>LOG_SOURCE_NAME</var>"]
  

Trend over time

Use the Trend Over Time graph to understand the distribution of raw logs over the time of your search. You can apply filters on the graph to look for parsed logs and raw logs. Click Expand more (or Expand less) to expand or collapse the graph.

Raw log results

When you run a raw log search, the results are a combination of UDM events and entities generated by the raw logs that match your searches, along with the raw logs. You can explore the search results further by clicking any of the results:

• UDM event or entity: If you click a UDM event or entity, Google SecOps shows any related events and entities, along with the raw log associated with that item.

• Raw log: If you click a raw log, Google SecOps shows you the entire raw log line, along with the source for that log.

Result limits and sampling

Raw log searches enforce the following result and query limits:

• Large-batch result limit: For recently ingested logs stored in batched storage before export to long-term columnar storage, there is a limit of 1,000 results returned from large log batches containing 1,000 or more events. Once logs are exported to long-term columnar storage, this 1,000-result batch limit doesn't apply.
• Total UI and CSV export limit: A single raw log search in the Google SecOps console or CSV export returns up to 10,000 total results across all queried log sources. If a query matches more than 10,000 entries, Google SecOps returns a representative sample of 10,000 results.
• API result limit: The Raw Log Search API (SearchRawLogs) enforces a maximum limit of 10,000 entries per query request (page_size).
• Search query length: The raw log search input field is limited to 150 characters.

When you search across multiple log sources simultaneously (leaving the Log Sources menu set to All), Google SecOps executes queries in parallel across distributed storage shards with protective query limits applied before in-memory filtering to maintain fast response times. Because of this distributed architecture, total returned result counts can vary between broad multi-source queries and single-source queries due to the following factors:

• Varying ingestion volumes: High-volume log sources generate large batches that reach the 1,000-result batch limit faster than lower-volume sources.
• Distributed batch sampling: Large batches containing 1,000 or more events are limited to 1,000 returned results before the overall 10,000-result query sampling limit is applied, which changes the proportion of sampled logs returned across queries.
• Recent and historical storage tiers: Recently ingested logs are queried from low-latency batched storage for immediate search availability, where the 1,000-result limit on large batches applies. Once logs are exported to long-term columnar storage, queries scan individual log entries directly without the 1,000-result batch limit (up to the 10,000-result total query limit), providing more consistent result counts for historical searches.

Note: Result count variations across multi-source queries are by design to maintain query responsiveness across large datasets and don't indicate data loss or an ingestion failure. To retrieve a complete, unsampled result set, filter your query to specific log sources and narrow the search time range to avoid hitting batch and overall query sampling limits.

Download raw log results

To download raw log results to a CSV file, in the Raw Logs results table, click More > Download as CSV.

By default, the CSV file always includes the Raw Log column. You can use the Columns button to open the Column Manager and remove or add the Timestamp and Type columns. If you add any other columns, you'll see them in the table, but they won't be downloaded in the CSV file.

To download UDM fields, run a UDM search, select the required columns, and then download as a CSV file.

If the Failed to execute 'showSaveFilePicker' error appears in Google Chrome, perform the following steps:

1. Go to Devices > Chrome > Settings.

2. Select Users & browsers.

3. Locate the File system write access setting.

4. Add prefix.backstory.chronicle.security to the list of allowed sites.

Regular expressions

You can use regular expressions in Google SecOps to search for and match sets of character strings in your security data. Regular expressions help to narrow your search using fragments of information rather than requiring an exact match.

To run a search using regular expression syntax:

• In SIEM Search, enter raw = /YOUR_REGEX/ in the search field and click Run Search.
• In the legacy Search bar:
  1. Enter a regular expression (4 to 66 characters long) in the Search field.
  2. Select the Run Query as Regex checkbox and click Search (or Update Search on the Raw Log Search results page).

The Google SecOps regular expression infrastructure is based on Google RE2, an open-source regular expression engine. Google SecOps uses the same regular expression syntax.

Note: For predictable performance, the RE2 engine intentionally omits certain advanced features, such as lookahead and lookbehind assertions. For more information, see RE2 syntax.

The following table highlights some of the common regular expression syntaxes you can use for your searches.

Any character	.
x number of any characters	{x}
Character class	[xyz]
Negated character class	[^xyz]
Alphanumeric (0-9A-Za-z)	[[:alnum:]]
Alphabetic (A-Za-z)	[[:alpha:]]
Digits (0-9)	[[:digit:]]
Lower case (a-z)	[[:lower:]]
Upper case (A-Z)	[[:upper:]]
Word characters (0-9A-Za-z_)	[[:word:]]
Hex digit (0-9A-Fa-f)	[[:xdigit:]]
Question mark symbol (?)	Matches zero or one occurrence of the preceding element.
Asterisk (*)	Matches zero or more occurrences of the preceding character or group.
Plus sign (+)	Matches one or more occurrences of the preceding character or group.

The following examples illustrate how you can use regular expressions to search data:

• goo.le\.com—Matches any string that starts with goo, followed by any single character, followed by le.com, such as google.com or goo0le.com.
• goo\w{3}\.com—Matches strings that start with goo, followed by exactly three word characters (\w), and ending with .com. Examples include google.com, goojle.com, or goodle.com.
• [[:digit:]]\.[[:alpha:]]—matches a string that has a single digit, followed by a dot (.), followed by a single alphabetic character, such as 34323.system, 23458.office, or 897.net.

Sample regular expressions to search for Windows logs

This section provides regular expression query strings you can use with Google SecOps Raw Log Search to find commonly monitored Windows events. These examples assume the Windows log messages are in JSON format.

For more information about commonly monitored Windows Event IDs, see Events to Monitor. The examples provided follow a similar pattern, described in these use cases.

Use Case: Return events with the EventID 1150
Regular expression string:	\"EventID\"\:\s*1150
Values Matched:	"EventID":1150

Use Case:Return events with an Event ID that is either 1150 or 1151
Regular expression string	(?:\"EventID\"\:\s*)(?:1150|1151)
Values Matched	"EventID":1150 and "EventID":1151

Use Case: Return events with an Event ID that is either 1150 or 1151, and with ThreadID 9092
Regular expression string	(?:\"EventID\"\:\s*)(?:1150|1151).*(?:\"ThreadID\"\:\s*9092)
Values Matched	"EventID":1150 <...any number of characters...> "ThreadID":9092 and "EventID":1151 <...any number of characters...> "ThreadID":9092

Find account management events

These regular expression query strings identify common account management events using the EventID attribute.

Type of Event	Regular Expression
User Account Created	"EventID\"\:\s*4720
User Account Enabled	"EventID\"\:\s*4722
User Account Disabled	"EventID\"\:\s*4725
User Account Deleted	"EventID\"\:\s*4726
User Rights Modification	"EventID\"\:\s*4703
Member Added to Security Enabled Global Group	"EventID\"\:\s*4728
Member Removed from Security Enabled Global Group	"EventID\"\:\s*4729
Security Enabled Global Group was Deleted	"EventID\"\:\s*4730

Find logon success events

These regular expression query strings identify types of successful logon events using the EventID and LogonType attributes.

Type of Event	Regular Expression
Logon Success	"EventID\"\:\s*4624
Logon Success - Interactive (LogonType=2)	"EventID\"\:\s*4624.*?LogonType\"\:\s*\"2\"
Logon Success - Batch Login (LogonType=4)	"EventID\"\:\s*4624.*?LogonType\"\:\s*\"4\"
Logon Success - Service Login (LogonType=5)	"EventID\"\:\s*4624.*?LogonType\"\:\s*\"5\"
Logon Success - RemoteInteractive Login (LogonType=10)	"EventID\"\:\s*4624.*?LogonType\"\:\s*\"10\"
Logon Success - Interactive, Batch, Service, or RemoteInteractive	(?:"EventID\"\:\s*4624.*?LogonType\"\:\s*\")(?:2|4|5|10)\"

Find logon failure events

These regular expression query strings identify types of failed logon events using the EventID and LogonType attributes.

Type of Event	Regular Expression
Logon Failure	"EventID\"\:\s*4625
Logon Failure - Interactive (LogonType=2)	"EventID\"\:\s*4625.*?LogonType\"\:\s*\"2\"
Logon Failure - Batch Login (LogonType=4)	"EventID\"\:\s*4625.*?LogonType\"\:\s*\"4\"
Logon Failure - Service Login (LogonType=5)	"EventID\"\:\s*4625.*?LogonType\"\:\s*\"5\"
Logon Failure - RemoteInteractive Login (LogonType=10)	"EventID\"\:\s*4625.*?LogonType\"\:\s*\"10\"
Logon Failure - Interactive, Batch, Service, or RemoteInteractive	(?:"EventID\"\:\s*4625.*LogonType\"\:\s*\")(?:2|4|5|10)\"

Find process, service and task events

These regular expression query strings identify certain process and service events using the EventID attribute.

Type of Event	Regular Expression
Process Start	"EventID\"\:\s*4688
Process Exit	"EventID\"\:\s*4689
Service Installed	"EventID\"\:\s*4697
New Service Created	"EventID\"\:\s*7045
Schedule Task Created	"EventID\"\:\s*4698

Find events related to object access

These regular expression query strings identify different types of process and service related events using the EventID attribute.

Type of Event	Regular Expression
Audit Log Cleared	"EventID\"\:\s*1102
Object Access Attempted	"EventID\"\:\s*4663
Share Accessed	"EventID\"\:\s*5140

Troubleshooting

Use the following table to resolve common raw log search issues:

Issue	Description	Fix
Inconsistent result counts across searches	Querying across multiple log sources (All) returns varying total result counts compared to single-source queries for recently ingested logs due to the 1,000-result limit on large batches (batches with 1,000 or more events) in batched storage and distributed sampling.	Select specific log sources from the Log Sources menu or filter using log_source = IN["LOG_SOURCE_NAME"], and narrow the search time range.
Truncated or sampled results (10,000-result limit reached)	The search query matched more than 10,000 total logs across the UI, CSV export, or SearchRawLogs API (page_size), returning only a 10,000-result sample.	Shorten the query time window (for example, to 1 hour) and add specific RE2 regular expressions or parsed = false filters.
Volumetric ingestion audit discrepancies	Raw log search result counts don't match total ingested log volumes because protective batch and shard limits are applied before in-memory filtering.	Use UDM Search, Statistics and aggregations in UDM Search, or ingestion health metrics to verify ingestion completeness and volumetric parity.
Query exceeds character limit	The search string exceeds the 150-character limit for raw log queries.	Simplify the regular expression or split the investigation into smaller, separate queries targeting individual log sources.

Need more help? Get answers from Community members and Google SecOps professionals.