Collect Oracle Cloud Infrastructure logs

Supported in:

Google secops SIEM

This document explains how to ingest Oracle Cloud Infrastructure logs to Google Security Operations using Google Cloud Storage.

Oracle Cloud Infrastructure (OCI) Audit service automatically records calls to all supported OCI public application programming interface (API) endpoints as log events. Log events recorded by the Audit service include API calls made by the OCI Console, Command Line Interface (CLI), Software Development Kits (SDK), your own custom clients, or other OCI services. The information in the logs includes the time the API activity occurred, the source and target of the activity, the type of action, and the type of response.

Before you begin

Make sure you have the following prerequisites:

• A Google SecOps instance
• A GCP project with Cloud Storage API enabled
• Permissions to create and manage GCS buckets
• Permissions to manage IAM policies on GCS buckets
• Permissions to create Cloud Run services, Pub/Sub topics, and Cloud Scheduler jobs
• An Oracle Cloud Infrastructure account with permissions to create and manage:
  • Service Connector Hub connectors
  • Functions applications and functions
  • Object Storage buckets
  • IAM policies
• Privileged access to the Oracle Cloud Console

Create a Google Cloud Storage bucket

1. Go to the Google Cloud Console.
2. Select your project or create a new one.
3. In the navigation menu, go to Cloud Storage > Buckets.
4. Click Create bucket.

5. Provide the following configuration details:

   Setting	Value
   Name your bucket	Enter a globally unique name (for example, oci-audit-logs-gcs)
   Location type	Choose based on your needs (Region, Dual-region, Multi-region)
   Location	Select the location (for example, us-central1)
   Storage class	Standard (recommended for frequently accessed logs)
   Access control	Uniform (recommended)
   Protection tools	Optional: Enable object versioning or retention policy

6. Click Create.

Configure Oracle Cloud Infrastructure to export Audit logs to GCS

Oracle Cloud Infrastructure does not support built-in export to Google Cloud Storage. You will use OCI Service Connector Hub with an OCI Function to forward Audit logs to GCS.

Create an OCI Function to forward logs to GCS

1. Sign in to the Oracle Cloud Console.
2. Open the navigation menu and select Developer Services > Functions > Applications.
3. Select the compartment where you want to create the function application.
4. Click Create Application.
5. Provide the following configuration details:
   • Name: Enter audit-logs-to-gcs-app.
   • VCN: Select a Virtual Cloud Network.
   • Subnets: Select a subnet with internet access.
6. Click Create.
7. After the application is created, click Getting Started and follow the instructions to set up your local development environment with the Fn CLI.

8. Create a new function directory on your local machine:

   mkdir oci-audit-to-gcs
   cd oci-audit-to-gcs
   

9. Initialize a Python function:

   fn init --runtime python oci-audit-to-gcs
   cd oci-audit-to-gcs
   

10. Replace the contents of func.py with the following code:

    import io
    import json
    import logging
    import os
    from fdk import response
    from google.cloud import storage
    from google.oauth2 import service_account
    from datetime import datetime
    
    # Configure logging
    logging.basicConfig(level=logging.INFO)
    logger = logging.getLogger()
    
    # Environment variables
    GCS_BUCKET = os.environ.get('GCS_BUCKET')
    GCS_PREFIX = os.environ.get('GCS_PREFIX', 'oci-audit-logs')
    GCS_CREDENTIALS_JSON = os.environ.get('GCS_CREDENTIALS_JSON')
    
    def handler(ctx, data: io.BytesIO = None):
        """
        OCI Function to forward Audit logs to GCS.
    
        Args:
            ctx: Function context
            data: Input data containing Audit log events
        """
    
        if not all([GCS_BUCKET, GCS_CREDENTIALS_JSON]):
            logger.error('Missing required environment variables: GCS_BUCKET or GCS_CREDENTIALS_JSON')
            return response.Response(
                ctx, response_data=json.dumps({"error": "Missing configuration"}),
                headers={"Content-Type": "application/json"}
            )
    
        try:
            # Parse input data
            body = json.loads(data.getvalue())
            logger.info(f"Received event: {json.dumps(body)}")
    
            # Extract log entries
            log_entries = []
            if isinstance(body, list):
                log_entries = body
            elif isinstance(body, dict):
                # Service Connector Hub sends data in specific format
                if 'data' in body:
                    log_entries = [body['data']] if isinstance(body['data'], dict) else body['data']
                else:
                    log_entries = [body]
    
            if not log_entries:
                logger.info("No log entries to process")
                return response.Response(
                    ctx, response_data=json.dumps({"status": "no_logs"}),
                    headers={"Content-Type": "application/json"}
                )
    
            # Initialize GCS client with service account credentials
            credentials_dict = json.loads(GCS_CREDENTIALS_JSON)
            credentials = service_account.Credentials.from_service_account_info(credentials_dict)
            storage_client = storage.Client(credentials=credentials, project=credentials_dict.get('project_id'))
            bucket = storage_client.bucket(GCS_BUCKET)
    
            # Write logs to GCS as NDJSON
            timestamp = datetime.utcnow().strftime('%Y%m%d_%H%M%S_%f')
            object_key = f"{GCS_PREFIX}/logs_{timestamp}.ndjson"
            blob = bucket.blob(object_key)
    
            ndjson = '\n'.join([json.dumps(entry, ensure_ascii=False) for entry in log_entries]) + '\n'
            blob.upload_from_string(ndjson, content_type='application/x-ndjson')
    
            logger.info(f"Wrote {len(log_entries)} records to gs://{GCS_BUCKET}/{object_key}")
    
            return response.Response(
                ctx, response_data=json.dumps({"status": "success", "records": len(log_entries)}),
                headers={"Content-Type": "application/json"}
            )
    
        except Exception as e:
            logger.error(f'Error processing logs: {str(e)}')
            return response.Response(
                ctx, response_data=json.dumps({"error": str(e)}),
                headers={"Content-Type": "application/json"},
                status_code=500
            )
    

11. Update requirements.txt with the following dependencies:

    fdk>=0.1.0
    google-cloud-storage>=2.0.0
    google-auth>=2.0.0
    

12. Deploy the function to OCI:

    fn -v deploy --app audit-logs-to-gcs-app
    

13. After deployment completes, note the function OCID. You will use it when configuring the Service Connector Hub.

Configure function environment variables

1. In the Oracle Cloud Console, go to Developer Services > Functions > Applications.
2. Click the application (audit-logs-to-gcs-app).
3. Click the function name (oci-audit-to-gcs).
4. Click Configuration.

5. Add the following configuration variables:

   Key	Value
   GCS_BUCKET	Your GCS bucket name (for example, oci-audit-logs-gcs)
   GCS_PREFIX	Prefix for log files (for example, oci-audit-logs)
   GCS_CREDENTIALS_JSON	JSON string of GCP service account key

6. Click Save changes.

Create a GCP service account for the OCI Function

The OCI Function needs a GCP service account to write to the GCS bucket.

1. In the GCP Console, go to IAM & Admin > Service Accounts.
2. Click Create Service Account.
3. Provide the following configuration details:
   • Service account name: Enter oci-function-gcs-writer.
   • Service account description: Enter Service account for OCI Function to write Audit logs to GCS.
4. Click Create and Continue.
5. In the Grant this service account access to project section, add the following role:
   1. Click Select a role.
   2. Search for and select Storage Object Admin.
6. Click Continue.
7. Click Done.
8. Click the newly created service account email.
9. Go to the Keys tab.
10. Click Add Key > Create new key.
11. Select JSON as the key type.
12. Click Create.
13. The JSON key file will be downloaded to your computer.
14. Open the JSON key file and copy its entire contents.
15. Return to the Oracle Cloud Console function configuration.

16. Paste the JSON contents into the GCS_CREDENTIALS_JSON configuration variable.

    Note: Store the service account key securely. Anyone with access to this key can write to your GCS bucket.

Grant IAM permissions on the GCS bucket

Grant the service account write permissions on the GCS bucket:

1. Go to Cloud Storage > Buckets.
2. Click your bucket name (oci-audit-logs-gcs).
3. Go to the Permissions tab.
4. Click Grant access.
5. Provide the following configuration details:
   • Add principals: Enter the service account email (oci-function-gcs-writer@PROJECT_ID.iam.gserviceaccount.com).
   • Assign roles: Select Storage Object Admin.
6. Click Save.

Create an OCI Service Connector Hub connector

1. Sign in to the Oracle Cloud Console.
2. Open the navigation menu and select Analytics & AI. Under Messaging, select Connector Hub.
3. Select the compartment where you want to create the service connector.
4. Click Create connector.

5. Provide the following configuration details:

   Connector information:

   Setting	Value
   Connector Name	Enter audit-logs-to-gcs-connector
   Description	Enter Forward OCI Audit logs to Google Cloud Storage
   Resource Compartment	Select the compartment

   Configure source:

   Setting	Value
   Source	Select Logging
   Compartment	Select the compartment containing audit logs
   Log Group	Select _Audit (default log group for audit logs)

6. Click + Another Log.

7. Select the audit log for your compartment (for example, _Audit_Include_Subcompartment).

   Note: If you want to capture audit logs from multiple compartments, click + Another Log to add additional log sources.

   Configure target:

   Setting	Value
   Target	Select Functions
   Function Compartment	Select the compartment containing the function
   Function Application	Select audit-logs-to-gcs-app
   Function	Select oci-audit-to-gcs

8. In the Configure task (optional) section, leave the default settings.

9. If prompted to create a policy (required for access to create or update a service connector), select Create.

10. Click Create.

Create IAM policy for the Service Connector Hub

The Service Connector Hub requires permissions to invoke the function.

1. In the Oracle Cloud Console, go to Identity & Security > Policies.
2. Select the compartment where you created the Service Connector Hub connector.
3. Click Create Policy.
4. Provide the following configuration details:
   • Name: Enter service-connector-functions-policy.
   • Description: Enter Allow Service Connector Hub to invoke Functions.
   • Compartment: Select the compartment.
5. In the Policy Builder section, toggle Show manual editor.

6. Enter the following policy statements:

   Allow any-user to use fn-function in compartment <compartment-name> where all {request.principal.type='serviceconnector'}
   Allow any-user to use fn-invocation in compartment <compartment-name> where all {request.principal.type='serviceconnector'}
   

   Note: Replace <compartment-name> with your compartment name.

7. Click Create.

Test the integration

1. Sign in to the Oracle Cloud Console.
2. Perform some actions that generate audit logs (for example, create or modify a resource).
3. Wait 2-5 minutes for logs to be processed.
4. Go to Cloud Storage > Buckets in the GCP Console.
5. Click your bucket name (oci-audit-logs-gcs).
6. Navigate to the prefix folder (oci-audit-logs/).

7. Verify that new .ndjson files are appearing in the bucket.

   Note: Each log file will be named with a timestamp in the format logs_YYYYMMDD_HHMMSS_microseconds.ndjson.

Retrieve the Google SecOps service account

Google SecOps uses a unique service account to read data from your GCS bucket. You must grant this service account access to your bucket.

Get the service account email

1. Go to SIEM Settings > Feeds.
2. Click Add New Feed.
3. Click Configure a single feed.
4. In the Feed name field, enter a name for the feed (for example, Oracle Cloud Audit Logs).
5. Select Google Cloud Storage V2 as the Source type.
6. Select Oracle Cloud Infrastructure Audit Logs as the Log type.

7. Click Get Service Account. A unique service account email is displayed, for example:

   [e-mail]
   

8. Copy this email address for use in the next step.

   Note: Each Google SecOps instance has a unique service account. Do not use service accounts from other documentation or examples.

Grant IAM permissions to the Google SecOps service account

The Google SecOps service account needs Storage Object Viewer role on your GCS bucket.

1. Go to Cloud Storage > Buckets.
2. Click your bucket name (oci-audit-logs-gcs).
3. Go to the Permissions tab.
4. Click Grant access.
5. Provide the following configuration details:
   • Add principals: Paste the Google SecOps service account email.
   • Assign roles: Select Storage Object Viewer.

6. Click Save.

   Note: If you plan to use the deletion option "Delete transferred files and empty directories," grant Storage Object Admin role instead of Storage Object Viewer.

Configure a feed in Google SecOps to ingest Oracle Cloud Infrastructure Audit logs

1. Go to SIEM Settings > Feeds.
2. Click Add New Feed.
3. Click Configure a single feed.
4. In the Feed name field, enter a name for the feed (for example, Oracle Cloud Audit Logs).
5. Select Google Cloud Storage V2 as the Source type.
6. Select Oracle Cloud Infrastructure Audit Logs as the Log type.
7. Click Next.

8. Specify values for the following input parameters:

   • Storage bucket URL: Enter the GCS bucket URI with the prefix path:

     gs://oci-audit-logs-gcs/oci-audit-logs/
     

   • Replace:

     • oci-audit-logs-gcs: Your GCS bucket name.
     • oci-audit-logs: Optional prefix/folder path where logs are stored (leave empty for root).

   • Examples:

     • Root bucket: gs://company-logs/
     • With prefix: gs://company-logs/oci-audit-logs/
     • With subfolder: gs://company-logs/oracle/audit/
   Note: Always include the trailing slash (/) at the end of the URI.
   • Source deletion option: Select the deletion option according to your preference:
     • Never delete files: Never deletes any files after transfers (recommended for testing).
     • Delete transferred files and empty directories: Deletes files and empty directories after successful transfer.
   Note: If you select a deletion option, the service account must have Storage Object Admin role instead of Storage Object Viewer. Update IAM permissions accordingly.
   • Maximum File Age (Days): Include files modified in the last number of days. Default is 180 days.
   • Asset namespace: The asset namespace.
   • Ingestion labels: The label to be applied to the events from this feed.

9. Click Next.

10. Review your new feed configuration in the Finalize screen, and then click Submit.

UDM mapping table

Log Field	UDM Mapping	Logic
authorization_field	additional.fields	Merged
compartment_id_field	additional.fields	Merged
compartment_name_field	additional.fields	Merged
content_length_field	additional.fields	Merged
content_type_field	additional.fields	Merged
event_grouping_id_field	additional.fields	Merged
tenant_id_label	additional.fields	Merged
data.message	metadata.description	Directly mapped
time	metadata.event_timestamp	Parsed as ISO8601
has_principal_hostname	metadata.event_type	Mapped: true → STATUS_UPDATE
oracle.tenantid	metadata.product_deployment_id	Directly mapped
type	metadata.product_event_type	Directly mapped
oracle.logid	metadata.product_log_id	Directly mapped
specversion	metadata.product_version	Directly mapped
data.request.action	network.http.method	Directly mapped
data.identity.userAgent	network.http.parsed_user_agent	Renamed/mapped
data.response.status	network.http.response_code	Renamed/mapped
ip_protocol_out	network.ip_protocol	Directly mapped
data.bytesOut	network.sent_bytes	Renamed/mapped
data.packets	network.sent_packets	Renamed/mapped
data.identity.consoleSessionId	network.session_id	Renamed/mapped
id	principal.asset.product_object_id	Directly mapped
source	principal.hostname	Directly mapped
data.identity.ipAddress	principal.ip	Merged
data.sourceAddress	principal.ip	Merged
data.sourcePort	principal.port	Renamed/mapped
compartmentid_label	principal.resource.attribute.labels	Merged
forward_label	principal.resource.attribute.labels	Merged
loggroupid_label	principal.resource.attribute.labels	Merged
vniccompartmentocid_label	principal.resource.attribute.labels	Merged
vnicocid_label	principal.resource.attribute.labels	Merged
vnicsubnetocid_label	principal.resource.attribute.labels	Merged
data.flowid	principal.resource.product_object_id	Directly mapped
credentials_label	principal.user.attribute.labels	Merged
data.identity.principalName	principal.user.user_display_name	Directly mapped
data.identity.principalId	principal.user.userid	Directly mapped
action	security_result.action	Merged
endTime_label	security_result.detection_fields	Merged
startTime_label	security_result.detection_fields	Merged
status_label	security_result.detection_fields	Merged
version_label	security_result.detection_fields	Merged
data.destinationAddress	target.ip	Merged
data.destinationPort	target.port	Renamed/mapped
data.request.path	target.url	Directly mapped
N/A	metadata.event_type	Constant: GENERIC_EVENT
N/A	metadata.product_name	Constant: ORACLE CLOUD AUDIT
N/A	metadata.vendor_name	Constant: ORACLE

Need more help? Get answers from Community members and Google SecOps professionals.