You can enhance the security of your AI applications by integrating Model Armor with Apigee. By using Apigee as an API gateway in front of your AI applications, you can use the capabilities of Model Armor to screen and sanitize both the inputs sent to your application and the responses received.
Overview
You can integrate Model Armor policies into your Apigee
API proxies to provide enhanced control and protection for your AI applications.
For example, you can use Apigee to enforce token-level quotas to limit
a user or application to a maximum number of tokens per day. Use
LLMTokenQuota
and PromptTokenLimit
for precise control over token usage. These policies help manage costs, stop
abuse, and stabilize traffic by preventing sudden spikes in consumption.
Before you begin
- Enable the Model Armor API in your project.
- Verify that billing is enabled for your Google Cloud project.
- Confirm that you have a Comprehensive environment available in your Apigee instance. You can only deploy Model Armor policies in Comprehensive environments.
- Confirm that you have an active Apigee organization provisioned.
To create and use Model Armor policies with Apigee, you must have specific roles. For the required roles, see Apigee roles.
How it works
The integration uses two main Apigee policies:
SanitizeUserPrompt: Add this policy to the request flow of your API proxy. It intercepts all incoming requests and sends them to Model Armor for inspection before they reach your AI application. For more information, seeSanitizeUserPromptpolicy.SanitizeModelResponse: Add this policy to the response flow. It intercepts the response from the AI application and sends it to Model Armor for inspection before sending the response to the client. For more information, seeSanitizeModelResponsepolicy.
Model Armor evaluates the content against the rules defined in your active Model Armor template. Based on the findings and the enforcement type set, Apigee allows, blocks, or redacts the request or response. If the request or response is redacted, extract the redacted data using flow variables and pass it to the LLM.
To get started, use the built-in Proxy with Model Armor proxy template in
the Apigee console. This template automatically adds optional
SanitizeUserPrompt and SanitizeModelResponse configurations.
Create an Apigee API proxy with Model Armor policies
- Create a service account to deploy the API proxy.
- Create an Apigee API proxy.
- Edit the
SanitizeUserPromptandSanitizeModelResponseXML. You must edit the XML of these policies for Model Armor to work. - Add Google authentication to the API proxy.
- Deploy the API proxy.
- Test the Model Armor policies.
Considerations
Consider the following when integrating Model Armor with Apigee:
- Regional availability: Model Armor isn't available in all regions. Ensure that Apigee and Model Armor are configured in the same region. See Locations for Model Armor.
- Quota: High volume traffic may require a quota increase for Model Armor. To request an adjustment, see Request a quota adjustment. If you need more than the default quota, contact Cloud Customer Care. For information on quotas related to Model Armor integrations with other services, see Manage quota.
- Token limits: Model Armor has token limits for processing prompts and responses, which vary by filter. Content exceeding these limits might not be fully scanned.
For more details on configuration and usage, see Get started with Apigee Model Armor policies.