Interaction

Response for InteractionService.CreateInteraction.

Fields
id string

Required. Output only. A unique identifier for the interaction completion.

status enum (Status)

Required. Output only. The status of the interaction.

created string

Required. Output only. The time at which the response was created in ISO 8601 format (YYYY-MM-DDThh:mm:ssZ).

updated string

Required. Output only. The time at which the response was last updated in ISO 8601 format (YYYY-MM-DDThh:mm:ssZ).

role
(deprecated) string

Output only. The role of the interaction.

outputs[]
(deprecated) object (Content)

Output only. Responses from the model.

systemInstruction string

System instruction for the interaction.

tools[] object (Tool)

A list of tool declarations the model may call during interaction.

usage object (Usage)

Output only. Statistics on the interaction request's token usage.

responseModalities[]
(deprecated) enum (ResponseModality)

The requested modalities of the response (TEXT, IMAGE, AUDIO).

responseMimeType
(deprecated) string

The mime type of the response. This is required if responseFormat is set.

previousInteractionId string

The id of the previous interaction, if any.

environmentId string

Output only. The environment id for the interaction. Only populated if environment config is set in the request.

steps[] object (Step)

Required. Output only. The steps that make up the interaction.

safety_settings[] object (SafetySetting)

Safety settings for the interaction.

labels map (key: string, value: string)

Optional. The labels with user-defined metadata for the request. It is used for billing and reporting only.

label keys and values can be no longer than 63 characters (Unicode codepoints) and can only contain lowercase letters, numeric characters, underscores, and dashes. International characters are allowed. label values are optional. label keys must start with a letter.

input Union type
The input for the interaction. input can be only one of the following:
contentList
(deprecated) object (ContentList)

The inputs for the interaction.

stringContent string

A string input for the interaction, it will be processed as a single text input.

turnList
(deprecated) object (TurnList)

The turns for the interaction.

stepList object (StepList)

Input only. The steps for the interaction.

content object (Content)

The content for the interaction.

response_format_config Union type
response_format_config can be only one of the following:
responseFormat
(deprecated) object (Value)

Enforces that the generated response is a JSON object that complies with the JSON schema specified in this field.

responseFormatList object (ResponseFormatList)
responseFormatSingleton object (ResponseFormat)
request_type Union type
The request type for the interaction. request_type can be only one of the following:
modelInteraction object (ModelInteraction)

Interaction for generating the completion using models.

agentInteraction object (AgentInteraction)

Interaction for generating the completion using agents.

environment Union type
The environment configuration for the interaction. environment can be only one of the following:
envId string

The environment id for the interaction. Can be 'remote' for default environment.

remoteEnvironment object (EnvironmentConfig)
localEnvironment object (LocalEnvironmentConfig)

The agent's environment lives on the client connection: its built-in environment operations (filesystem ops and running commands) are yielded to the client to execute, instead of running in a server-managed sandbox. Mutually exclusive with remoteEnvironment. (Independent of any client-declared function tools, which are always executed on the client regardless of this field.)

JSON representation 
{
  "id": string,
  "status": enum (Status),
  "created": string,
  "updated": string,
  "role": string,
  "outputs": [
    {
      object (Content)
    }
  ],
  "systemInstruction": string,
  "tools": [
    {
      object (Tool)
    }
  ],
  "usage": {
    object (Usage)
  },
  "responseModalities": [
    enum (ResponseModality)
  ],
  "responseMimeType": string,
  "previousInteractionId": string,
  "environmentId": string,
  "steps": [
    {
      object (Step)
    }
  ],
  "safety_settings": [
    {
      object (SafetySetting)
    }
  ],
  "labels": {
    string: string,
    ...
  },

  // input
  "contentList": {
    object (ContentList)
  },
  "stringContent": string,
  "turnList": {
    object (TurnList)
  },
  "stepList": {
    object (StepList)
  },
  "content": {
    object (Content)
  }
  // Union type

  // response_format_config
  "responseFormat": {
    object (Value)
  },
  "responseFormatList": {
    object (ResponseFormatList)
  },
  "responseFormatSingleton": {
    object (ResponseFormat)
  }
  // Union type

  // request_type
  "modelInteraction": {
    object (ModelInteraction)
  },
  "agentInteraction": {
    object (AgentInteraction)
  }
  // Union type

  // environment
  "envId": string,
  "remoteEnvironment": {
    object (EnvironmentConfig)
  },
  "localEnvironment": {
    object (LocalEnvironmentConfig)
  }
  // Union type
}

TurnList

A list of Turns.

Fields
turns[] object (Turn)
JSON representation 
{
  "turns": [
    {
      object (Turn)
    }
  ]
}

Turn

Fields
role string

The originator of this turn. Must be user for input or model for model output.

content Union type
content can be only one of the following:
contentList object (ContentList)

The content of the turn. An array of Content objects.

contentString string

The content of the turn. A single string.

JSON representation 
{
  "role": string,

  // content
  "contentList": {
    object (ContentList)
  },
  "contentString": string
  // Union type
}

StepList

A list of Steps.

Fields
steps[] object (Step)

The steps of the list.

JSON representation 
{
  "steps": [
    {
      object (Step)
    }
  ]
}

ResponseFormatList

Fields
responseFormats[] object (ResponseFormat)
JSON representation 
{
  "responseFormats": [
    {
      object (ResponseFormat)
    }
  ]
}

ResponseFormat

Fields
type Union type
type can be only one of the following:
audio object (AudioResponseFormat)
text object (TextResponseFormat)
image object (ImageResponseFormat)
video object (VideoResponseFormat)
structValue object (Struct)

Multi-discriminator values is already enabled in GAOS

JSON representation 
{

  // type
  "audio": {
    object (AudioResponseFormat)
  },
  "text": {
    object (TextResponseFormat)
  },
  "image": {
    object (ImageResponseFormat)
  },
  "video": {
    object (VideoResponseFormat)
  },
  "structValue": {
    object (Struct)
  }
  // Union type
}

AudioResponseFormat

Configuration for audio output format.

Fields
mimeType enum (MimeType)

The MIME type of the audio output.

delivery enum (Delivery)

The delivery mode for the audio output.

sampleRate integer

Sample rate in Hz.

bitRate integer

Bit rate in bits per second (bps). Only applicable for compressed formats (MP3, Opus).

JSON representation 
{
  "mimeType": enum (MimeType),
  "delivery": enum (Delivery),
  "sampleRate": integer,
  "bitRate": integer
}

MimeType

Supported MIME types for audio output.

Enums
TYPE_UNSPECIFIED Default value. This value is unused.
TYPE_MP3 MP3 audio format.
TYPE_OGG_OPUS OGG Opus audio format.
TYPE_L16 Raw PCM (L16) audio format.
TYPE_WAV WAV audio format.
TYPE_ALAW A-law audio format.
TYPE_MULAW Mu-law audio format.

Delivery

Delivery mode for audio output.

Enums
DELIVERY_UNSPECIFIED Default value. This value is unused.
INLINE Audio data is returned inline in the response.
URI Audio data is returned as a URI.

TextResponseFormat

Configuration for text output format.

Fields
mimeType enum (MimeType)

The MIME type of the text output.

schema object (Struct)

The JSON schema that the output should conform to. Only applicable when mimeType is application/json.

JSON representation 
{
  "mimeType": enum (MimeType),
  "schema": {
    object (Struct)
  }
}

MimeType

Supported MIME types for text output.

Enums
TYPE_UNSPECIFIED Default value. This value is unused.
TYPE_APPLICATION_JSON JSON output format.
TYPE_TEXT_PLAIN Plain text output format.

ImageResponseFormat

Configuration for image output format.

Fields
mimeType enum (MimeType)

The MIME type of the image output.

delivery enum (Delivery)

The delivery mode for the image output.

aspectRatio enum (AspectRatio)

The aspect ratio for the image output.

imageSize enum (ImageSize)

The size of the image output.

JSON representation 
{
  "mimeType": enum (MimeType),
  "delivery": enum (Delivery),
  "aspectRatio": enum (AspectRatio),
  "imageSize": enum (ImageSize)
}

MimeType

Supported MIME types for image output.

Enums
TYPE_UNSPECIFIED Default value. This value is unused.
TYPE_JPEG JPEG image format.

Delivery

Delivery mode for image output.

Enums
DELIVERY_UNSPECIFIED Default value. This value is unused.
INLINE Image data is returned inline in the response.
URI Image data is returned as a URI.

AspectRatio

Supported aspect ratios for image output.

Enums
ASPECT_RATIO_UNSPECIFIED Default value. This value is unused.
ASPECT_RATIO_ONE_BY_ONE 1:1 aspect ratio.
ASPECT_RATIO_TWO_BY_THREE 2:3 aspect ratio.
ASPECT_RATIO_THREE_BY_TWO 3:2 aspect ratio.
ASPECT_RATIO_THREE_BY_FOUR 3:4 aspect ratio.
ASPECT_RATIO_FOUR_BY_THREE 4:3 aspect ratio.
ASPECT_RATIO_FOUR_BY_FIVE 4:5 aspect ratio.
ASPECT_RATIO_FIVE_BY_FOUR 5:4 aspect ratio.
ASPECT_RATIO_NINE_BY_SIXTEEN 9:16 aspect ratio.
ASPECT_RATIO_SIXTEEN_BY_NINE 16:9 aspect ratio.
ASPECT_RATIO_TWENTY_ONE_BY_NINE 21:9 aspect ratio.
ASPECT_RATIO_ONE_BY_EIGHT 1:8 aspect ratio.
ASPECT_RATIO_EIGHT_BY_ONE 8:1 aspect ratio.
ASPECT_RATIO_ONE_BY_FOUR 1:4 aspect ratio.
ASPECT_RATIO_FOUR_BY_ONE 4:1 aspect ratio.

ImageSize

Supported image sizes for image output.

Enums
IMAGE_SIZE_UNSPECIFIED Default value. This value is unused.
IMAGE_SIZE_FIVE_TWELVE 512px image size.
IMAGE_SIZE_ONE_K 1K image size.
IMAGE_SIZE_TWO_K 2K image size.
IMAGE_SIZE_FOUR_K 4K image size.

VideoResponseFormat

Configuration for video output format.

Fields
delivery enum (Delivery)

The delivery mode for the video output.

gcsUri string

The Cloud Storage URI to store the video output. Required for Vertex if delivery mode is URI.

aspectRatio enum (AspectRatio)

The aspect ratio for the video output.

duration string (Duration format)

The duration for the video output.

A duration in seconds with up to nine fractional digits, ending with 's'. Example: "3.5s".

JSON representation 
{
  "delivery": enum (Delivery),
  "gcsUri": string,
  "aspectRatio": enum (AspectRatio),
  "duration": string
}

Delivery

Delivery mode for video output.

Enums
DELIVERY_UNSPECIFIED Default value. This value is unused.
INLINE Video data is returned inline in the response.
URI Video data is returned as a URI.

AspectRatio

Supported aspect ratios for video output.

Enums
ASPECT_RATIO_UNSPECIFIED Default value. This value is unused.
ASPECT_RATIO_SIXTEEN_BY_NINE 16:9 aspect ratio.
ASPECT_RATIO_NINE_BY_SIXTEEN 9:16 aspect ratio.

ModelInteraction

Interaction for generating the completion using models.

Fields
model string

The name of the Model used for generating the completion.

generationConfig object (GenerationConfig)

Input only. Configuration parameters for the model interaction.

JSON representation 
{
  "model": string,
  "generationConfig": {
    object (GenerationConfig)
  }
}

GenerationConfig

Configuration parameters for model interactions.

Fields
temperature number

Controls the randomness of the output.

topP number

The maximum cumulative probability of tokens to consider when sampling.

seed integer

Seed used in decoding for reproducibility.

stopSequences[] string

A list of character sequences that will stop output interaction.

thinkingLevel enum (ThinkingLevel)

The level of thought tokens that the model should generate.

thinkingSummaries enum (ThinkingSummaries)

Whether to include thought summaries in the response.

maxOutputTokens integer

The maximum number of tokens to include in the response.

speechConfig[] object (SpeechConfig)

Configuration for speech interaction.

imageConfig
(deprecated) object (ImageConfig)

Configuration for image interaction.

videoConfig object (VideoConfig)

Configuration for video generation.

transcriptionConfig object (TranscriptionConfig)

Optional. Configuration for speech recognition (transcription). If present, ASR is enabled.

tool_choice Union type
The tool choice configuration. tool_choice can be only one of the following:
toolChoiceMode enum (ToolChoiceType)

The mode of the tool choice.

toolChoiceConfig object (ToolChoiceConfig)

The config for the tool choice.

JSON representation 
{
  "temperature": number,
  "topP": number,
  "seed": integer,
  "stopSequences": [
    string
  ],
  "thinkingLevel": enum (ThinkingLevel),
  "thinkingSummaries": enum (ThinkingSummaries),
  "maxOutputTokens": integer,
  "speechConfig": [
    {
      object (SpeechConfig)
    }
  ],
  "imageConfig": {
    object (ImageConfig)
  },
  "videoConfig": {
    object (VideoConfig)
  },
  "transcriptionConfig": {
    object (TranscriptionConfig)
  },

  // tool_choice
  "toolChoiceMode": enum (ToolChoiceType),
  "toolChoiceConfig": {
    object (ToolChoiceConfig)
  }
  // Union type
}

ToolChoiceType

The type of tool choice.

Enums
TOOL_CHOICE_TYPE_UNSPECIFIED Default value. This value is unused.
AUTO Auto tool choice.
ANY Any tool choice.
NONE No tool choice.
VALIDATED Validated tool choice.

ToolChoiceConfig

The tool choice configuration containing allowed tools.

Fields
allowedTools object (AllowedTools)

The allowed tools.

JSON representation 
{
  "allowedTools": {
    object (AllowedTools)
  }
}

AllowedTools

The configuration for allowed tools.

Fields
mode enum (ToolChoiceType)

The mode of the tool choice.

tools[] string

The names of the allowed tools.

JSON representation 
{
  "mode": enum (ToolChoiceType),
  "tools": [
    string
  ]
}

ThinkingLevel

The level of thought tokens that the model should generate.

Enums
THINKING_LEVEL_UNSPECIFIED Default value. This value is unused.
THINKING_LEVEL_MINIMAL Little to no thinking.
THINKING_LEVEL_LOW Low thinking level.
THINKING_LEVEL_MEDIUM Medium thinking level.
THINKING_LEVEL_HIGH High thinking level.

ThinkingSummaries

Whether to include thought summaries in the response.

Enums
THINKING_SUMMARIES_UNSPECIFIED Default value. This value is unused.
THINKING_SUMMARIES_AUTO Auto thinking summaries.
THINKING_SUMMARIES_NONE No thinking summaries.

SpeechConfig

The configuration for speech interaction.

Fields
voice string

The voice of the speaker.

language string

The language of the speech.

speaker string

The speaker's name, it should match the speaker name given in the prompt.

JSON representation 
{
  "voice": string,
  "language": string,
  "speaker": string
}

ImageConfig

The configuration for image interaction.

Fields
aspectRatio string

The aspect ratio of the image to generate. Supported aspect ratios: 1:1, 2:3, 3:2, 3:4, 4:3, 9:16, 16:9, 21:9.

If not specified, the model will choose a default aspect ratio based on any reference images provided.

imageSize string

Specifies the size of generated images. Supported values are 1K, 2K, 4K. If not specified, the model will use default value 1K.

JSON representation 
{
  "aspectRatio": string,
  "imageSize": string
}

VideoConfig

Configuration options for video generation.

Fields
task enum (Task)

Optional task mode for video generation. If not specified, the model automatically determines the appropriate mode based on the provided text prompt and input media.

JSON representation 
{
  "task": enum (Task)
}

Task

Supported video generation tasks.

Enums
TASK_UNSPECIFIED Unspecified task. The task is inferred from the input prompt and media.
TEXT_TO_VIDEO Generates video solely from a text prompt.
IMAGE_TO_VIDEO Generates video from one or two source images. The first image defines the starting frame, and the optional second image defines the ending frame.
REFERENCE_TO_VIDEO Generates video using reference media (such as images, audio, or video).
EDIT Modifies an existing input video.

TranscriptionConfig

Configuration for speech recognition (transcription).

Fields
timestampGranularities[] string

Optional. The granularity of timestamps to include in the transcription output. Supported values: "word". If empty, no timestamps are generated.

diarizationMode string

Optional. Configures speaker diarization. Supported values: "speaker".

languageCodes[] string

Optional. BCP-47 language codes providing hints about the languages present in the audio. If omitted or empty, defaults to automatic language detection.

adaptationPhrases[]
(deprecated) string

Optional. A list of phrases to bias the ASR model towards.

customVocabulary[] string

Optional. A list of custom vocabulary phrases to bias the speech recognition model toward recognizing specific terms.

JSON representation 
{
  "timestampGranularities": [
    string
  ],
  "diarizationMode": string,
  "languageCodes": [
    string
  ],
  "adaptationPhrases": [
    string
  ],
  "customVocabulary": [
    string
  ]
}

AgentInteraction

Interaction for generating the completion using agents.

Fields
agent string

The name of the Agent used for generating the completion.

agent_config Union type
Parameters for the agent interaction. agent_config can be only one of the following:
dynamicConfig object (DynamicAgentConfig)
deepResearchConfig object (DeepResearchAgentConfig)
codeMenderConfig object (CodeMenderAgentConfig)
antigravityConfig object (AntigravityAgentConfig)

Antigravity agent configuration. This configuration is session-level settings that are passed to the agent runtime on a per-request basis.

JSON representation 
{
  "agent": string,

  // agent_config
  "dynamicConfig": {
    object (DynamicAgentConfig)
  },
  "deepResearchConfig": {
    object (DeepResearchAgentConfig)
  },
  "codeMenderConfig": {
    object (CodeMenderAgentConfig)
  },
  "antigravityConfig": {
    object (AntigravityAgentConfig)
  }
  // Union type
}

DynamicAgentConfig

Configuration for dynamic agents.

Fields
config object (Struct)

For agents that are not supported statically in the API definition.

JSON representation 
{
  "config": {
    object (Struct)
  }
}

DeepResearchAgentConfig

Configuration for the Deep Research agent.

Fields
thinkingSummaries enum (ThinkingSummaries)

Whether to include thought summaries in the response.

visualization enum (VisualizationMode)

Whether to include visualizations in the response.

collaborativePlanning boolean

Enables human-in-the-loop planning for the Deep Research agent. If set to true, the Deep Research agent will provide a research plan in its response. The agent will then proceed only if the user confirms the plan in the next turn.

enableBigqueryTool boolean

Enables bigquery tool for the Deep Research agent.

JSON representation 
{
  "thinkingSummaries": enum (ThinkingSummaries),
  "visualization": enum (VisualizationMode),
  "collaborativePlanning": boolean,
  "enableBigqueryTool": boolean
}

VisualizationMode

Enum for visualization mode. Eventually we will support an interactive mode where the user can choose whether to include HTML visualizations in the response.

Enums
UNSPECIFIED The default visualization mode. Will default to AUTO.
OFF Do not include visualizations.
AUTO Automatically include visualizations.

CodeMenderAgentConfig

Configuration for the CodeMender agent.

Fields
session_id string

Parameter for grouping multiple interactions that belong to the same CodeMender session.

session_config object (SessionConfig)

Optional session-specific configurations to override default agent behavior.

model string

The name of the model to use for the CodeMender agent. One CodeMender session will only use one model.

request Union type
CodeMender's request type. Set exactly one of find_request/fix_request only on the first round to start a session; on subsequent rounds (e.g. submitting tool results), leave this unset and identify the session via session_id. This oneof is intentionally not a subtype_source discriminator so it can be omitted on resume rounds. request can be only one of the following:
find_request object (FindRequest)

Parameters for finding vulnerabilities.

fix_request object (FixRequest)

Parameters for fixing vulnerabilities.

JSON representation 
{
  "session_id": string,
  "session_config": {
    object (SessionConfig)
  },
  "model": string,

  // request
  "find_request": {
    object (FindRequest)
  },
  "fix_request": {
    object (FixRequest)
  }
  // Union type
}

FindRequest

Request parameters specific to FIND sessions, used for discovering vulnerabilities in a codebase.

Fields
source_files[] object (FileContent)

A list of source files to provide as context for the scan.

finding_id string

The identifier of a specific finding to verify. This is primarily used in VERIFY mode to focus the agent's execution-based validation on a single vulnerability.

description string

Additional context or custom instructions provided by the user to guide the vulnerability analysis.

mode enum (Mode)

The mode of the find session.

JSON representation 
{
  "source_files": [
    {
      object (FileContent)
    }
  ],
  "finding_id": string,
  "description": string,
  "mode": enum (Mode)
}

FileContent

Content of a single file in the codebase.

Fields
path string

The relative path of the file from the project root.

content string

The UTF-8 encoded text content of the file.

JSON representation 
{
  "path": string,
  "content": string
}

Mode

Defines the depth and thoroughness of the find session.

Enums
MODE_UNSPECIFIED Default value. This value is unused.
MODE_SCAN Fast scan using only the initial classifier.
MODE_VERIFY Performs classification followed by detailed investigation.

FixRequest

Request parameters specific to FIX sessions, used for generating and validating security patches.

Fields
source_files[] object (FileContent)

A list of source files providing context for the remediation. These files are typically the ones containing the identified vulnerability.

finding_id string

The identifier of the specific security finding to be remediated. This id maps to a previously discovered vulnerability.

description string

Additional context or custom instructions provided by the user to guide the patch generation process.

JSON representation 
{
  "source_files": [
    {
      object (FileContent)
    }
  ],
  "finding_id": string,
  "description": string
}

SessionConfig

The configuration of CodeMender sessions.

Fields
max_rounds integer

The maximum number of interaction rounds the agent is allowed to perform before reaching a timeout.

JSON representation 
{
  "max_rounds": integer
}

AntigravityAgentConfig

Configuration for the Antigravity agent runtime. Provides server-side control over the agent's execution environment and tool configuration.

Fields
model string

The model to use for agent reasoning.

maxTotalTokens string (int64 format)

Max total tokens for the agent run.

JSON representation 
{
  "model": string,
  "maxTotalTokens": string
}

EnvironmentConfig

Configuration for a custom environment.

Fields
sources[] object (Source)
environmentId string

Optional. The environment id for the interaction. If specified, the request will update the existing environment instead of creating a new one.

network Union type
Network configuration for the environment. network can be only one of the following:
network_allowlist object (EnvironmentNetworkEgressAllowlist)

Allow only specific domains.

network_mode enum (NetworkMode)

Network egress mode.

JSON representation 
{
  "sources": [
    {
      object (Source)
    }
  ],
  "environmentId": string,

  // network
  "network_allowlist": {
    object (EnvironmentNetworkEgressAllowlist)
  },
  "network_mode": enum (NetworkMode)
  // Union type
}

EnvironmentNetworkEgressAllowlist

Network egress configuration for the environment.

Fields
allowlist[] object (EgressRule)

List of allowed domains and their configurations.

JSON representation 
{
  "allowlist": [
    {
      object (EgressRule)
    }
  ]
}

EgressRule

A network egress rule that controls which external domains the environment is allowed to reach. Each rule identifies a target domain and, optionally, a set of HTTP headers to inject into every matching outbound request.

Fields
domain string

The domain pattern to match for this rule. Use an exact hostname (e.g., github.com), a wildcard prefix (e.g., *.googleapis.com), or * to match all domains.

transform map (key: string, value: string)

headers to inject into requests matching this rule. Key: header name (e.g., "Authorization"). value: header value (e.g., "Bearer your-token").

JSON representation 
{
  "domain": string,
  "transform": {
    string: string,
    ...
  }
}

NetworkMode

Network egress mode for non-allowlist configurations.

Enums
NETWORK_MODE_UNSPECIFIED Default value. Unused.
DISABLED All network egress is blocked.

Source

A source to be mounted into the environment.

Fields
type enum (Type)
source string

The source of the environment. For Cloud Storage, this is the Cloud Storage path. For GitHub, this is the GitHub path.

target string

Where the source should appear in the environment.

content string

The inline content if type is INLINE.

encoding string

Optional encoding for inline content (e.g. base64).

JSON representation 
{
  "type": enum (Type),
  "source": string,
  "target": string,
  "content": string,
  "encoding": string
}

Type

Enums
TYPE_UNSPECIFIED
GCS A Cloud Storage bucket.
INLINE Inline content.
REPOSITORY A generic repository. The protocol prefix in the source URL identifies the provider (e.g., github://, gcs://).
SKILL_REGISTRY A skill resource from the Skill Registry service. Skill: projects/{project}/locations/{location}/skills/{skill} SkillRevision: projects/{project}/locations/{location}/skills/{skill}/revisions/{revision} Support mounting all skills under a project: projects/{project}/locations/{location}/skills.

LocalEnvironmentConfig

This type has no fields.

Configuration for an environment that lives on the client connection rather than in a server-managed sandbox.

When set (via Interaction.local_environment), the agent's filesystem and shell are treated as living on the client: the agent's built-in environment operations (e.g. reading/listing/editing files and running commands) are suspended on the server and yielded back to the client to execute, with their results returned on a subsequent turn. This is mutually exclusive with a server-managed EnvironmentConfig (remoteEnvironment), since the environment is either on the client or in a server sandbox, never both.

This governs only the agent's built-in environment. client-declared function tools are always executed on the client regardless of this field.

Tool

A tool that can be used by the model.

Fields
type Union type
The tool to use. type can be only one of the following:
function object (Function)

A function that can be used by the model.

codeExecution object (CodeExecution)

A tool that can be used by the model to execute code.

urlContext object (UrlContext)

A tool that can be used by the model to fetch URL context.

computerUse object (ComputerUse)

Tool to support the model interacting directly with the computer.

mcpServer object (McpServer)

A MCPServer is a server that can be called by the model to perform actions.

googleMaps object (GoogleMaps)

A tool that can be used by the model to search Google Maps.

retrieval object (Retrieval)

A tool that can be used by the model to retrieve files.

JSON representation 
{

  // type
  "function": {
    object (Function)
  },
  "codeExecution": {
    object (CodeExecution)
  },
  "urlContext": {
    object (UrlContext)
  },
  "computerUse": {
    object (ComputerUse)
  },
  "mcpServer": {
    object (McpServer)
  },
  "googleSearch": {
    object (GoogleSearch)
  },
  "fileSearch": {
    object (FileSearch)
  },
  "googleMaps": {
    object (GoogleMaps)
  },
  "retrieval": {
    object (Retrieval)
  }
  // Union type
}

Function

A tool that can be used by the model.

Fields
name string

The name of the function.

description string

A description of the function.

parameters object (Value)

The JSON Schema for the function's parameters.

JSON representation 
{
  "name": string,
  "description": string,
  "parameters": {
    object (Value)
  }
}

CodeExecution

This type has no fields.

A tool that can be used by the model to execute code.

UrlContext

This type has no fields.

A tool that can be used by the model to fetch URL context.

ComputerUse

A tool that can be used by the model to interact with the computer.

Fields
environment enum (Environment)

The environment being operated.

excludedPredefinedFunctions[] string

The list of predefined functions that are excluded from the model call.

enablePromptInjectionDetection boolean

Whether enable the prompt injection detection check on computer-use request.

disabledSafetyPolicies[] enum (SafetyPolicy)

Optional. disabled safety policies for computer use.

JSON representation 
{
  "environment": enum (Environment),
  "excludedPredefinedFunctions": [
    string
  ],
  "enablePromptInjectionDetection": boolean,
  "disabledSafetyPolicies": [
    enum (SafetyPolicy)
  ]
}

Environment

Represents the environment being operated, such as a web browser.

Enums
ENVIRONMENT_UNSPECIFIED Defaults to browser.
BROWSER Operates in a web browser.
MOBILE Operates in a mobile environment.
DESKTOP Operates in a desktop environment.

SafetyPolicy

Enums
SAFETY_POLICY_UNSPECIFIED Unspecified safety policy.
FINANCIAL_TRANSACTIONS Safety policy for financial transactions.
SENSITIVE_DATA_MODIFICATION Safety policy for sensitive data modification.
COMMUNICATION_TOOL Safety policy for communication tools (e.g. Gmail, Chat, Meet).
ACCOUNT_CREATION Safety policy for account creation.
DATA_MODIFICATION Safety policy for data modification.
LEGAL_TERMS_AND_AGREEMENTS Safety policy for legal terms and agreements.

McpServer

A MCPServer is a server that can be called by the model to perform actions.

Fields
name string

The name of the MCPServer.

url string

The full URL for the MCPServer endpoint. Example: "https://api.example.com/mcp"

headers map (key: string, value: string)

Optional: Fields for authentication headers, timeouts, etc., if needed.

allowedTools[] object (AllowedTools)

The allowed tools.

JSON representation 
{
  "name": string,
  "url": string,
  "headers": {
    string: string,
    ...
  },
  "allowedTools": [
    {
      object (AllowedTools)
    }
  ]
}

GoogleSearch

A tool that can be used by the model to search Google.

Fields
searchTypes[] enum (SearchType)

The types of search grounding to enable.

JSON representation 
{
  "searchTypes": [
    enum (SearchType)
  ]
}

FileSearch

A tool that can be used by the model to search files.

Fields
fileSearchStoreNames[] string

The file search store names to search.

topK integer

The number of semantic retrieval chunks to retrieve.

metadataFilter string

metadata filter to apply to the semantic retrieval documents and chunks.

JSON representation 
{
  "fileSearchStoreNames": [
    string
  ],
  "topK": integer,
  "metadataFilter": string
}

GoogleMaps

A tool that can be used by the model to call Google Maps.

Fields
enableWidget boolean

Whether to return a widget context token in the tool call result of the response.

latitude number

The latitude of the user's location.

longitude number

The longitude of the user's location.

JSON representation 
{
  "enableWidget": boolean,
  "latitude": number,
  "longitude": number
}

Retrieval

A tool that can be used by the model to retrieve files.

Fields
retrievalTypes[] enum (RetrievalType)

The types of file retrieval to enable.

vertex_ai_search_config object (VertexAISearchConfig)

Used to specify configuration for VertexAISearch.

exa_ai_search_config object (ExaAISearchConfig)

Used to specify configuration for ExaAISearch.

parallel_ai_search_config object (ParallelAISearchConfig)

Used to specify configuration for ParallelAISearch.

rag_store_config object (RagStoreConfig)

Used to specify configuration for RagStore.

JSON representation 
{
  "retrievalTypes": [
    enum (RetrievalType)
  ],
  "vertex_ai_search_config": {
    object (VertexAISearchConfig)
  },
  "exa_ai_search_config": {
    object (ExaAISearchConfig)
  },
  "parallel_ai_search_config": {
    object (ParallelAISearchConfig)
  },
  "rag_store_config": {
    object (RagStoreConfig)
  }
}

VertexAISearchConfig

Used to specify configuration for VertexAISearch.

Fields
engine string

Optional. Used to specify Agent Platform Search engine.

datastores[] string

Optional. Used to specify Agent Platform Search datastores.

JSON representation 
{
  "engine": string,
  "datastores": [
    string
  ]
}

ExaAISearchConfig

Used to specify configuration for ExaAISearch.

Fields
api_key string

Required. The API key for ExaAiSearch.

custom_config object (Struct format)

Optional. This field can be used to pass any parameter from the Exa.ai Search API.

JSON representation 
{
  "api_key": string,
  "custom_config": {
    object
  }
}

ParallelAISearchConfig

Used to specify configuration for ParallelAISearch.

Fields
api_key string

Optional. The API key for ParallelAiSearch.

custom_config object (Struct format)

Optional. Custom configs for ParallelAiSearch.

JSON representation 
{
  "api_key": string,
  "custom_config": {
    object
  }
}

RagStoreConfig

Use to specify configuration for RAG Store.

Fields
rag_resources[] object (RagResource)

Optional. The representation of the rag source.

similarity_top_k
(deprecated) integer

Optional. Number of top k results to return from the selected corpora.

vector_distance_threshold
(deprecated) number

Optional. Only return results with vector distance smaller than the threshold.

rag_retrieval_config object (RagRetrievalConfig)

Optional. The retrieval config for the Rag query.

JSON representation 
{
  "rag_resources": [
    {
      object (RagResource)
    }
  ],
  "similarity_top_k": integer,
  "vector_distance_threshold": number,
  "rag_retrieval_config": {
    object (RagRetrievalConfig)
  }
}

RagResource

The definition of the Rag resource.

Fields
rag_corpus string

Optional. RagCorpora resource name.

rag_file_ids[] string

Optional. ragFileId. The files should be in the same ragCorpus set in ragCorpus field.

JSON representation 
{
  "rag_corpus": string,
  "rag_file_ids": [
    string
  ]
}

RagRetrievalConfig

Specifies the context retrieval config.

Fields
top_k integer

Optional. The number of contexts to retrieve.

filter object (Filter)

Optional. Config for filters.

ranking object (Ranking)

Optional. Config for ranking and reranking.

JSON representation 
{
  "top_k": integer,
  "hybrid_search": {
    object (HybridSearch)
  },
  "filter": {
    object (Filter)
  },
  "ranking": {
    object (Ranking)
  }
}

HybridSearch

Config for Hybrid Search.

Fields
alpha number

Optional. Alpha value controls the weight between dense and sparse vector search results.

JSON representation 
{
  "alpha": number
}

Filter

Config for filters.

Fields
metadata_filter string

Optional. String for metadata filtering.

vector_db_threshold Union type
Filter contexts retrieved from the vector DB based on either vector distance or vector similarity. vector_db_threshold can be only one of the following:
vector_distance_threshold number

Optional. Only returns contexts with vector distance smaller than the threshold.

vector_similarity_threshold number

Optional. Only returns contexts with vector similarity larger than the threshold.

JSON representation 
{
  "metadata_filter": string,

  // vector_db_threshold
  "vector_distance_threshold": number,
  "vector_similarity_threshold": number
  // Union type
}

Ranking

Config for ranking and reranking.

Fields
ranking_config Union type
Config options for ranking. ranking_config can be only one of the following:
rank_service object (RankService)

Optional. Config for Rank service.

JSON representation 
{

  // ranking_config
  "rank_service": {
    object (RankService)
  }
  // Union type
}

RankService

Config for Rank service.

Fields
model_name string

Optional. The model name of the rank service.

JSON representation 
{
  "model_name": string
}

SafetySetting

A safety setting that affects the safety-blocking behavior.

A SafetySetting consists of a harm category and a threshold for that category.

Fields
type enum (HarmCategory)

Required. The type of harm category to be blocked.

threshold enum (HarmBlockThreshold)

Required. The threshold for blocking content. If the harm probability exceeds this threshold, the content will be blocked.

method enum (HarmBlockMethod)

Optional. The method for blocking content. If not specified, the default behavior is to use the probability score.

JSON representation 
{
  "type": enum (HarmCategory),
  "threshold": enum (HarmBlockThreshold),
  "method": enum (HarmBlockMethod)
}

HarmCategory

Harm categories that can be detected in user input and model responses.

Enums
HARM_CATEGORY_UNSPECIFIED Default value. This value is unused.
HARM_CATEGORY_HATE_SPEECH Content that promotes violence or incites hatred against individuals or groups based on certain attributes.
HARM_CATEGORY_DANGEROUS_CONTENT Content that promotes, facilitates, or enables dangerous activities.
HARM_CATEGORY_HARASSMENT Abusive, threatening, or content intended to bully, torment, or ridicule.
HARM_CATEGORY_SEXUALLY_EXPLICIT Content that contains sexually explicit material.
HARM_CATEGORY_CIVIC_INTEGRITY

Deprecated: Election filter is not longer supported. The harm category is civic integrity.
HARM_CATEGORY_IMAGE_HATE Images that contain hate speech.
HARM_CATEGORY_IMAGE_DANGEROUS_CONTENT Images that contain dangerous content.
HARM_CATEGORY_IMAGE_HARASSMENT Images that contain harassment.
HARM_CATEGORY_IMAGE_SEXUALLY_EXPLICIT Images that contain sexually explicit content.
HARM_CATEGORY_JAILBREAK Prompts designed to bypass safety filters.

HarmBlockThreshold

Thresholds for blocking content based on harm probability.

Enums
HARM_BLOCK_THRESHOLD_UNSPECIFIED The harm block threshold is unspecified.
BLOCK_LOW_AND_ABOVE Block content with a low harm probability or higher.
BLOCK_MEDIUM_AND_ABOVE Block content with a medium harm probability or higher.
BLOCK_ONLY_HIGH Block content with a high harm probability.
BLOCK_NONE Do not block any content, regardless of its harm probability.
OFF Turn off the safety filter entirely.

HarmBlockMethod

The method for blocking content.

Enums
HARM_BLOCK_METHOD_UNSPECIFIED The harm block method is unspecified.
SEVERITY The harm block method uses both probability and severity scores.
PROBABILITY The harm block method uses the probability score.