Integrate UrlScan.io with Google SecOps

This document provides guidance on how to integrate urlscan.io with Google SecOps.

This document explains how to integrate UrlScan.io with Google Security Operations.

Use cases

The UrlScan.io integration uses Google SecOps capabilities to support the following use cases:

Automated URL analysis: Automatically submit suspicious URLs to UrlScan.io to determine risk levels and retrieve screenshots for visual verification during incident triage.
Threat intelligence enrichment: Enrich alerts by querying UrlScan.io for detailed metadata on IP addresses, domains, and URLs, providing analysts with immediate context on ASN, server types, and malicious scores.
Proactive historical scan search: Search existing public and private scans associated with indicators of compromise to identify historical malicious activity and patterns.
Deep-dive forensics and reporting: Retrieve comprehensive scan details using specific scan IDs to perform in-depth analysis of cookies, request counts, and related links for complex security investigations.

Before you begin

To authenticate the connection between Google SecOps and UrlScan.io, you must provide a valid API key.

You can obtain and manage your API keys within the Profile section of your urlscan.io account.

Integration parameters

The urlscan.io integration requires the following parameters:

Parameter Description

Parameter	Description
`Api Key`	Required. The unique API key used to authenticate with the urlscan.io service.
`Verify SSL`	Optional. If selected, the integration validates the SSL certificate when connecting to the UrlScan.io server.

Api Key

Required.

The unique API key used to authenticate with the urlscan.io service.

Verify SSL

Optional.

If selected, the integration validates the SSL certificate when connecting to the UrlScan.io server.

For instructions about how to configure an integration in Google SecOps, see Configure integrations.

You can make changes at a later stage, if needed. After you configure an integration instance, you can use it in playbooks. For more information about how to configure and support multiple instances, see Supporting multiple instances.

Actions

For more information about actions, see Respond to pending actions from Your Workdesk and Perform a manual action.

Ping

Test Connectivity.

Parameters

N/A

Use cases

N/A

Run On

This action runs on all entities.

Action Results

Entity Enrichment

Insights

N/A

Script Result

Script Result Name	Value Options	Example
is_success	True/False	is_success:False

JSON Result

N/A

URL Check

Submit a URL to be scanned and get the scan details.

Parameters

Parameter Name	Type	Is Mandatory	Default Value	Description
Visibility	DDL possible: public, unlisted, private.	No	public	Scans on urlscan.io have one of three visibility levels, make sure to use the appropriate level for your submission.
Threshold	integer	No	-1	Mark entity as suspicious if the score of verdicts is equal or above the given threshold. Default is -1, in this case, we consider every scanned url as suspicious.
Create Insight	Boolean	No	Yes	If enabled, action will create an insight containing information about entities.
Only Suspicious Insight	Boolean	No	No	If enabled, action will only create insight for suspicious entities. Note: "Create Insight" parameter needs to be enabled.
Add Screenshot To Insight	Boolean	No	No	If enabled, action will add a screenshot of the website to the insight, if it's available.

Use cases

N/A

Run On

This action runs on the following entities:

IP Address
Domain
URL

Action Results

Entity Enrichment

Name	Key
real_url	tasks/url
visibility	visibility
requests_count	len(data/requests)
cookies	CSV of data/cookies/name
related_links	CSV of data/links/href
main_country	page/country
main_domain	page/domain
main_ip	page/ip
main_asn	page/asnname
main_server	page/server
related_ips_count	len(lists/ips)
related_domains_count	len(lists/domains)
related_countries	CSV lists/countries
overall_score	verdicts/overall/score
categories	verdicts/overall/categories
tags	verdicts/overall/tags
malicious	verdicts/overall/malicious

Insights

N/A

Script Result

Script Result Name	Value Options	Example
is_success	True/False	is_success:False

JSON Result

[
    {
        "EntityResult":
        {
            "task":
            {
                "domURL": "https://urlscan.io/dom/7e9cb8cb-82ce-4ef7-881a-8958d95fbd1b/",
                "screenshotURL": "https://urlscan.io/screenshots/7e9cb8cb-82ce-4ef7-881a-8958d95fbd1b.png",
                "uuid": "7e9cb8cb-82ce-4ef7-881a-8958d95fbd1b",
                "url": "http://markossolomon.com/f1q7qx.php",
                "visibility": "public",
                "source": "12a3ddaf",
                "time": "2019-01-31T15:19:55.267Z",
                "reportURL": "https://urlscan.io/result/7e9cb8cb-82ce-4ef7-881a-8958d95fbd1b/",
                "userAgent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.87 Safari/537.36",
                "method": "api"
            },
            "stats":
            {
                "malicious": 0,
                "uniqCountries": 1,
                "totalLinks": 3,
                "secureRequests": 14,
                "securePercentage": 93,
                "adBlocked": 0,
                "IPv6Percentage": 50
            },
            "page":
            {
                "city": "Los Angeles",
                "domain": "markossolomon.com",
                "asn": "AS22612",
                "url": "http://markossolomon.com/f1q7qx.php",
                "ip": "1.1.1.1",
                "asnname": "NAMECHEAP-NET - Namecheap, Inc., US",
                "server": "nginx",
                "country": "US",
                "ptr": ""
            },
            "lists":
            {
                "linkDomains": ["www.namecheap.com",
                                "ap.www.namecheap.com"],
                "countries": ["US"],
                "asns": ["22612"],
                "servers": ["cloudflare",
                            "nginx"],
                "ips": ["198.54.117.244"],
                "urls": ["http://markossolomon.com/f1q7qx.php"],
                "domains": ["nc-img.com"],
                "hashes": ["f31c0889d28c7d713f237a8cea8cfbc5cb4cba63fad767666cce2bbc99746d1a"],
                "certificates": [{
                    "subjectName": "nc-img.com",
                    "validFrom": 1534204800,
                    "validTo": 1565827199,
                    "issuer": "COMODO RSA Domain Validation Secure Server CA"
                }]
            }},
        "Entity": "HTTP://MARKOSSOLOMON.COM/F1Q7QX.PHP"
    }
]

Search For Scans

Search for urlscan.io existing scans by attributes such as domains, IPs, Autonomous System (AS) numbers, hashes, etc. The action will find public scans performed by anyone as well as unlisted and private scans performed by you or your teams.

Parameters

Parameter Name	Type	Is Mandatory	Default Value	Description
Max Scans	Integer	No	100	Number of scans to return per entity. Default: 100, Max: 10000 (depending on subscription).

Run On

This action runs on the following entities:

IP address
Hostnames
URLs
Filename
Hashes
Domain

Action Results

Entity Enrichment

N/A

Insights

N/A

Script Result

Script Result Name	Value Options	Example
is_success	True/False	is_success:False

JSON Result

{"entity_identifier": "www.unitedneighborsfcu.com",
"entity_results":[
  {
        "indexedAt": "2020-12-09T12:16:43.329Z",
        "task": {
            "visibility": "public",
            "method": "automatic",
            "domain": "www.unitedneighborsfcu.com",
            "time": "2020-12-09T12:16:23.168Z",
            "source": "certstream-suspicious",
            "uuid": "96310829-fed4-4d61-9fb0-39eb2952719f",
            "url": "https://www.unitedneighborsfcu.com"
        },
        "stats": {
            "uniqIPs": 6,
            "consoleMsgs": 0,
            "uniqCountries": 3,
            "dataLength": 1938842,
            "encodedDataLength": 1568193,
            "requests": 28
        },
        "page": {
            "country": "US",
            "server": "Microsoft-IIS/10.0",
            "domain": "www.unitedneighborsfcu.com",
            "ip": "8.21.114.55",
            "mimeType": "text/html",
            "asnname": "LEVEL3, US",
            "asn": "AS3356",
            "url": "https://www.unitedneighborsfcu.com/",
            "status": "200"
        },
        "_id": "96310829-fed4-4d61-9fb0-39eb2952719f",
        "sort": [1607516183168, "96310829-fed4-4d61-9fb0-39eb2952719f"],
        "result": "https://urlscan.io/api/v1/result/96310829-fed4-4d61-9fb0-39eb2952719f/",
        "screenshot": "https://urlscan.io/screenshots/96310829-fed4-4d61-9fb0-39eb2952719f.png"
  }
  ]
}

Case Wall

Result Type	Value / Description	Type
Output message*	The action should not fail nor stop a playbook execution: if find scans for some of the entities (is_success = true): print "Successfully listed scans for the following entities:\n".format(entity.identifier) If didn't find scans for some of the entities (is_success = true): print "Action wasn't able to list scans for the following entities:\n".format(entity.identifier) If didn't find scans for all of the entities(is_success = false): print "Action wasn't able to list scans for the available entities". If no entities: print "No suitable entities were found in the current scope. The action should fail and stop a playbook execution: if fatal error, like wrong credentials, no connection to server, other: print "Error executing action "Search for Scans". Reason: {0}''.format(error.Stacktrace).	General
Case Wall Table	Title: "{entity identifier} - Search Results" Columns: Scan ID URL Scan Date Size IPS Unique Countries Country Scan Type	General
Case Wall Link	Title: "urlscan.io Web Report + (entity ID).	General
Case Wall attachment	Will contain the screenshot.	General

Get Scan Full Details

Get Scan Full Details by scan ID

Parameters

Parameter Name	Type	Is Mandatory	Default Value	Description
Scan ID	String	Yes	N/A	Get scan report using the scan ID. Comma-separated values.

Run On

This action doesn't run on entities.

Action Results

Entity Enrichment

N/A

Insights

N/A

Script Result

Script Result Name	Value Options	Example
is_success	True/False	is_success:False

JSON Result

['Effective URL'] = response['page']['url']

Case Wall

Result Type	Value / Description	Type
Output message*	The action should not fail nor stop a playbook execution: if find some scan ids (is_success = true): print "Successfully fetched results for the following scans: {scan ids} If didn't find some (is_success = true): print "Action wasn't able to fetch results for the following scans: {scan ids} If didn't find all (is_success = false): print "Action wasn't able to fetch results. The provided scan ids are not available using urlscan.io" The action should fail and stop a playbook execution: if fatal error, like wrong credentials, no connection to server, other: print "Error executing action "Get Scan Full Details". Reason: {0}''.format(error.Stacktrace)	General
Case Wall link	Title: "urlscan.io Web Report + (Scan ID).	General
Case Wall attachment	Will contain the screenshot.	General

Result Type

Value / Description

Type

Output message*

The action should not fail nor stop a playbook execution:

if find some scan ids (is_success = true): print "Successfully fetched results for the following scans: {scan ids}
If didn't find some (is_success = true): print "Action wasn't able to fetch results for the following scans: {scan ids}
If didn't find all (is_success = false): print "Action wasn't able to fetch results. The provided scan ids are not available using urlscan.io"

The action should fail and stop a playbook execution:

if fatal error, like wrong credentials, no connection to server, other: print "Error executing action "Get Scan Full Details". Reason: {0}''.format(error.Stacktrace)

General

Case Wall link

Title: "urlscan.io Web Report + (Scan ID).

General

Case Wall attachment

Will contain the screenshot.

General

Need more help? Get answers from Community members and Google SecOps professionals.