AI Protection overview

Premium and Enterprise service tiers (requires organization-level activation)

AI Protection helps you manage the security posture of your AI workloads by detecting threats and helping you to mitigate risks to your AI asset inventory. This document provides a general overview of AI Protection, including its benefits and several key concepts. AI Protection is available with organization-level activations of Security Command Center.

For the Premium and Enterprise service tiers, when Security Command Center is activated at the organization level, AI Protection helps provide a comprehensive view of AI security across your entire Google Cloud environment. The AI Protection dashboard within the Google Cloud console displays a consistent set of widgets and features, with data aggregated from all projects and resources within the organization.

Capabilities of AI Protection

AI Protection helps you manage threats and risks to your AI systems in the following ways:

• Assess your AI asset inventory: Assess and understand your AI systems and AI assets, including the following:
  • Models
  • Data sources
  • Endpoints
  • Agents (Preview)
  • Model Context Protocol (MCP) servers that are cataloged in Agent Registry (Preview). Discovery of MCP servers requires the App Hub API (apphub.googleapis.com) to be enabled in each project that hosts an MCP server.
• Identify vulnerabilities: Identify software vulnerabilities (CVEs) in agentic workloads deployed with Agent Engine.
• Identify risks: Identify agentic risks and their impact on the ecosystem based on Attack Path Simulation and predefined security graph rules, with agents and MCP servers as high-value resources.
• Detect over-privileged agents: Detect Agent Engine agents that are granted excessive permissions. (Preview)
• Manage risks and compliance: Proactively manage risks to your AI assets and verify that your AI deployments adhere to relevant security standards.
• Mitigate legal and financial risks: Reduce the financial, reputational, and legal risks associated with security breaches and regulatory noncompliance.
• Detect and manage threats: Detect and respond to potential threats to your AI systems and assets in a timely manner.
• View one dashboard: Manage all of your AI-related risks and threats from one centralized dashboard.

Use cases for AI Protection

AI Protection helps organizations enhance their security by identifying and mitigating threats and risks related to AI systems and sensitive data. The following use cases are examples of how AI Protection can be used in different organizations:

• Financial services institution: customer financial data

  A large financial services institution uses AI models that process sensitive financial data.

  • Challenge: Processing highly sensitive financial data with AI models entails several risks, including the risk of data breaches, data exfiltration during training or inference, and vulnerabilities in the underlying AI infrastructure.
  • Use case: AI Protection continuously monitors AI workflows for suspicious activity, works to detect unauthorized data access and anomalous model behavior, performs sensitive data classification, and aids in improving your compliance with regulations such as PCI DSS and GDPR.

• Healthcare provider: patient privacy and compliance

  A major healthcare provider manages electronic health records and uses AI for diagnostics and treatment planning, dealing with Protected Health Information (PHI).

  • Challenge: PHI analyzed by AI models is subject to strict regulations like HIPAA. Risks include accidental PHI exposure through misconfigurations or malicious attacks that target AI systems for patient data.
  • Use case: AI Protection identifies and alerts on potential HIPAA violations, detects unauthorized PHI access by models or users, flags vulnerable and potentially misconfigured AI services, and monitors for data leakage.

• Manufacturing and robotics company: proprietary intellectual property

  A manufacturing company specializing in advanced robotics and automation relies heavily on AI for optimizing production lines and robotic control, with vital intellectual property (IP) embedded within its AI algorithms and manufacturing data.

  • Challenge: Proprietary AI algorithms and sensitive operational data are vulnerable to theft from insider threats or external adversaries, potentially leading to competitive disadvantage or operational disruption.
  • Use case: AI Protection monitors for unauthorized access to AI models and code repositories, detects attempts to exfiltrate trained models and unusual data access patterns, and flags vulnerabilities in AI development environments to prevent IP theft.

Event Threat Detection rules for Vertex AI assets

The following Event Threat Detection rules run detections on Vertex AI assets:

• Persistence: New AI API Method
• Persistence: New Geography for AI Service
• Privilege Escalation: Anomalous Impersonation of Service Account for AI Admin Activity
• Privilege Escalation: Anomalous Service Account Impersonator for AI Data Access
• Privilege Escalation: Anomalous Multistep Service Account Delegation for AI Admin Activity
• Privilege Escalation: Anomalous Multistep Service Account Delegation for AI Data Access
• Privilege Escalation: Anomalous Service Account Impersonator for AI Admin Activity
• Initial Access: Dormant Service Account Activity in AI Service
• Persistence: IAM Anomalous Grant to Agentic Identity
• Credential Access: Agentic Identity Credential Used Outside of Google Cloud
• Persistence: Sensitive AI Permission Added to Custom Role
• Persistence: Sensitive Role Granted by AI Agent
• Persistence: Sensitive Role Granted to External AI Agent
• Defense Evasion: Project Level Token Creator Role Granted to AI Agent
• Defense Evasion: Folder Level Token Creator Role Granted to AI Agent
• Defense Evasion: Organization Level Token Creator Role Granted to AI Agent

For more information about Event Threat Detection, see Event Threat Detection overview.

Agent Platform Threat Detection for Vertex AI Agent Engine

Preview

This feature is subject to the "Pre-GA Offerings Terms" in the General Service Terms section of the Service Specific Terms. Pre-GA features are available "as is" and might have limited support. For more information, see the launch stage descriptions.

Agent Platform Threat Detection provides runtime threat detection for agents deployed to Vertex AI Agent Engine. It monitors running agents for potential attacks and generates findings in Security Command Center.

Agent Platform Threat Detection can generate findings for Vertex AI Agent Engine including the following categories:

• Command and Control: Steganography Tool Detected
• Credential Access: Find Google Cloud Credentials
• Credential Access: GPG Key Reconnaissance
• Credential Access: Search Private Keys or Passwords
• Defense Evasion: Base64 ELF File Command Line
• Defense Evasion: Base64 Encoded Python Script Executed
• Defense Evasion: Base64 Encoded Shell Script Executed
• Defense Evasion: Launch Code Compiler Tool In Container
• Execution: Netcat Remote Code Execution in Container
• Execution: Possible Arbitrary Command Execution through CUPS (CVE-2024-47177)
• Execution: Possible Remote Command Execution Detected
• Execution: Program Run with Disallowed HTTP Proxy Env
• Execution: Socat Reverse Shell Detected
• Execution: Suspicious OpenSSL Shared Object Loaded
• Exfiltration: Launch Remote File Copy Tools in Container
• Impact: Detect Malicious Cmdlines
• Impact: Remove Bulk Data from Disk
• Impact: Suspicious crypto mining activity using the Stratum Protocol
• Privilege Escalation: Abuse of Sudo For Privilege Escalation (CVE-2019-14287)
• Privilege Escalation: Polkit Local Privilege Escalation Vulnerability (CVE-2021-4034)
• Privilege Escalation: Sudo Potential Privilege Escalation (CVE-2021-3156)
• Execution: Malicious Python executed
• Execution: Container Escape
• Execution: Kubernetes Attack Tool Execution
• Execution: Local Reconnaissance Tool Execution
• Impact: Malicious Script Executed
• Impact: Malicious URL Observed
• Execution: Unexpected Child Shell

AI Protection framework

AI Protection uses a framework that includes specific cloud controls that are deployed automatically in detective mode. Detective mode means that the cloud control is applied to the defined resources for monitoring purposes. Any violations are detected and alerts are generated. You use frameworks and cloud controls to define your AI Protection requirements and apply those requirements to your Google Cloud environment. AI Protection includes the Default framework, which defines recommended baseline controls for AI Protection. When you enable AI Protection, the default framework is automatically applied to the Google Cloud organization in detective mode.

If required, you can make copies of the framework to create custom AI Protection frameworks. You can add the cloud controls to your custom frameworks and apply the custom frameworks to the organization, folders, or projects. For example, you can create custom frameworks that apply specific jurisdictional controls to specific folders to ensure that data within those folders stays within a particular geographical region.

Note: You can't assign AI Protection frameworks to applications.

Cloud controls in the default AI Protection framework

For more information about the cloud controls that AI Protection framework uses, see Google Recommended AI Essentials - Vertex AI.

Supported functional areas for AI Protection

This section defines functional areas that AI Protection can help secure.

• AI workloads: AI application workloads range from internal tools aimed at improving employee productivity to consumer-facing solutions designed to enhance the user experience and drive business. Examples include AI agents, virtual assistants, conversational AI chatbots, and personalized recommendations.
• AI agents: AI agents are AI systems that can perceive their environment, make decisions, and take actions to achieve specific goals.
• AI models: AI models are classified into foundation AI models, fine-tuned AI models, standard first-party AI models, and custom AI models. Examples include Gemini, Llama, translation models, and custom models for specific tasks.
• AI assets: AI assets contribute to machine learning operation pipelines and are used by AI workloads. Types of AI assets include the following:
  • Declarative AI assets: AI lifecycle management tools, such as Vertex AI, track these assets.
  • Inferred AI assets: General-purpose assets, such as compute and storage assets, used to process AI data or workloads.
  • Model-as-a-Service (API only): Assets that have programmatic calls into first-party or third-party AI models.

Use the AI Security dashboard

The AI Security dashboard lets you visualize your organization's AI asset inventory and review proposed mitigations for risks and threats.

Access the AI Security dashboard

To access the AI security dashboard, in the Google Cloud console, go to the Risk overview > AI security page:

Go to AI security

Note: For the dashboard to populate with data, you need one of the IAM roles in the Required roles section.

For more information, see AI Security dashboard.

Understand risk management for AI systems

This section provides information about potential risks that are associated with AI systems. You can view the top risks in your AI inventory.

You can click any issue to open a details pane that provides a visualization of the issue.

View AI threats

This section provides insights into threats associated with AI systems. You can view the top 5 recent threats associated with your AI resources.

On this page, you can do the following:

• Click View all to see threats that are associated with your AI resources.
• Click any threat to see further details about the threat.

Visualize your AI inventory

You can view a visualization of your AI inventory on the dashboard that provides a summary of the projects that involve generative AI, the first-party and third-party models in active use, and the datasets that are used in training the third-party models.

On this page, you can do the following:

• To view the inventory details page, click any of the nodes in the visualization.
• To view a detailed listing of individual assets (such as foundational models and custom-built models), click the tooltip.
• To open a detailed view of the model, click the model. This view displays details such as the endpoints where the model is hosted and the dataset used to train the model. If Sensitive Data Protection is enabled, the datasets view also displays whether the dataset contains any sensitive data.

Review AI framework findings summary

This section helps you assess and manage findings from the AI framework and data security policies, and includes the following:

• Findings: This section displays a summary of findings generated by AI security policies and data security policies. Click View all findings or click the count against each finding category to view details about the finding. Click a finding to display additional information about that finding.
• Sensitive data in Vertex AI datasets: This section displays a summary of the findings based on sensitive data in datasets as reported by Sensitive Data Protection. For more information, see Introduction to Vertex AI.

Examine Model Armor findings

A graph shows the total number of prompts or responses that Model Armor scanned and the number of issues that Model Armor detected. In addition, the graph displays summary statistics for various types of detected issues, such as prompt injection and jailbreak detection, and sensitive data detection. For agentic workloads, Model Armor can be configured on Agent Gateway (Preview) to screen prompts and responses to and from agents.

This information is populated based on the metrics that Model Armor publishes to Cloud Monitoring. For more information, see Model Armor overview.

What's next

• Learn how to configure AI Protection.
• To assess risk, access dashboard data.