This document shows you how to configure various capabilities of
Gemini models when using Gemini Live API. You can configure tool use
such as function calling and grounding, and live audio capabilities such as
affective dialog and proactive audio.


> [!NOTE]
> To learn more,
> run the following notebooks in the environment of your choice:
>
> - "Introduction to the Live API (WebSocket)":
>
>   [![](https://docs.cloud.google.com/static/vertex-ai/images/colab-logo-32px.png)Open in Colab](https://colab.research.google.com/github/GoogleCloudPlatform/generative-ai/blob/main/gemini/multimodal-live-api/intro_multimodal_live_api.ipynb)
>
>
>   \|
>
>   [![](https://docs.cloud.google.com/static/vertex-ai/images/colab-enterprise-logo-32px.png)Open in Colab Enterprise](https://console.cloud.google.com/agent-platform/colab/import/https%3A%2F%2Fraw.githubusercontent.com%2FGoogleCloudPlatform%2Fgenerative-ai%2Fmain%2Fgemini%2Fmultimodal-live-api%2Fintro_multimodal_live_api.ipynb)
>
>
>   \|
>
>   [![](https://docs.cloud.google.com/static/vertex-ai/images/vertex-ai-workbench-logo-32px.png)Open
>   in Agent Platform Workbench](https://console.cloud.google.com/agent-platform/workbench/deploy-notebook?download_url=https://raw.githubusercontent.com/GoogleCloudPlatform/generative-ai/main/gemini/multimodal-live-api/intro_multimodal_live_api.ipynb)
>
>
>   \|
>
>   [![](https://docs.cloud.google.com/static/vertex-ai/images/github-logo-32px.png)View on GitHub](https://github.com/GoogleCloudPlatform/generative-ai/blob/main/gemini/multimodal-live-api/intro_multimodal_live_api.ipynb)
> - "Introduction to the Live API (Google Gen AI SDK)":
>
>   [![](https://docs.cloud.google.com/static/vertex-ai/images/colab-logo-32px.png)Open in Colab](https://colab.research.google.com/github/GoogleCloudPlatform/generative-ai/blob/main/gemini/multimodal-live-api/intro_multimodal_live_api_genai_sdk.ipynb)
>
>
>   \|
>
>   [![](https://docs.cloud.google.com/static/vertex-ai/images/colab-enterprise-logo-32px.png)Open in Colab Enterprise](https://console.cloud.google.com/agent-platform/colab/import/https%3A%2F%2Fraw.githubusercontent.com%2FGoogleCloudPlatform%2Fgenerative-ai%2Fmain%2Fgemini%2Fmultimodal-live-api%2Fintro_multimodal_live_api_genai_sdk.ipynb)
>
>
>   \|
>
>   [![](https://docs.cloud.google.com/static/vertex-ai/images/vertex-ai-workbench-logo-32px.png)Open
>   in Agent Platform Workbench](https://console.cloud.google.com/agent-platform/workbench/deploy-notebook?download_url=https://raw.githubusercontent.com/GoogleCloudPlatform/generative-ai/main/gemini/multimodal-live-api/intro_multimodal_live_api_genai_sdk.ipynb)
>
>
>   \|
>
>   [![](https://docs.cloud.google.com/static/vertex-ai/images/github-logo-32px.png)View on GitHub](https://github.com/GoogleCloudPlatform/generative-ai/blob/main/gemini/multimodal-live-api/intro_multimodal_live_api_genai_sdk.ipynb)

## Configure tool use

Several tools are compatible with various versions of
Gemini Live API-supported models, including:

- [Function calling](https://docs.cloud.google.com/gemini-enterprise-agent-platform/models/live-api/configure-gemini-capabilities.md.txt#function-calling)
- [Model Context Protocol (MCP) tools](https://docs.cloud.google.com/gemini-enterprise-agent-platform/models/live-api/configure-gemini-capabilities.md.txt#mcp-tools)
- [Grounding with Google Search](https://docs.cloud.google.com/gemini-enterprise-agent-platform/models/live-api/configure-gemini-capabilities.md.txt#grounding-google-search)

To enable a particular tool for usage in returned responses, include the name of
the tool in the `tools` list when you initialize the model. The following
sections provide examples of how to use each of the built-in tools in your code.

**Important:** The Gemini API doesn't support combining search tools
(such as `googleSearch`) with non-search tools (such as function calling or
the Gemini Enterprise Agent Platform RAG Engine `retrieval` tool) in
the same setup request. Multiple tools are supported only when they are all
search tools.

### Function calling

Use [function calling](https://docs.cloud.google.com/gemini-enterprise-agent-platform/models/tools/function-calling.md.txt) when you want the model to
interact with external systems or APIs that you manage. Use this for tasks like
checking a database, sending an email, or interacting with a custom API.
Gemini Live API supports asynchronous function calling. For more information,
see [Asynchronous function calling with
Gemini Live API](https://docs.cloud.google.com/gemini-enterprise-agent-platform/models/live-api/asynchronous-function-calling).

The model generates a function call, and your application executes the code and
sends the results back to the model.

All functions must be declared at the start of the session by sending tool
definitions as part of the `LiveConnectConfig` message.

To enable function calling, include `function_declarations` in the `tools` list
in the setup message:

### Python

```python
import asyncio

from google import genai
from google.genai.types import (
    Content,
    LiveConnectConfig,
    Part,
)

# Initialize the client.
client = genai.Client(
    enterprise=True,
    project="GOOGLE_CLOUD_PROJECT",  # Replace with your project ID
    location="LOCATION",  # Replace with your location
)

MODEL_ID = "gemini-3.8-live"


def get_current_weather(location: str) -> str:
    """Example method. Returns the current weather.

    Args:
        location: The city and state, e.g. San Francisco, CA
    """
    weather_map: dict[str, str] = {
        "Boston, MA": "snowing",
        "San Francisco, CA": "foggy",
        "Seattle, WA": "raining",
        "Austin, TX": "hot",
        "Chicago, IL": "windy",
    }
    return weather_map.get(location, "unknown")


async def main():
    config = LiveConnectConfig(
        response_modalities=["AUDIO"],
        tools=[get_current_weather],
    )

    async with client.aio.live.connect(
        model=MODEL_ID,
        config=config,
    ) as session:
        text_input = "Get the current weather in Boston."
        print(f"Input: {text_input}")

        await session.send_realtime_input(text=text_input)
        )

        async for message in session.receive():
            if message.tool_call:
                function_responses = []
                for function_call in message.tool_call.function_calls:
                    print(f"FunctionCall > {function_call}")
                    # Execute the tool and send the response back to the model.
                    result = get_current_weather(**function_call.args)
                    function_responses.append(
                        {
                            "name": function_call.name,
                            "response": {"result": result},
                            "id": function_call.id,
                        }
                    )
                if function_responses:
                    await session.send_tool_response(function_responses=function_responses)


if __name__ == "__main__":
    asyncio.run(main())
  
```

For examples using function calling in system instructions, see the
[best practices example](https://docs.cloud.google.com/gemini-enterprise-agent-platform/models/live-api/best-practices.md.txt#best-practices-example).

### Model Context Protocol (MCP) tools

The Google Gen AI SDK doesn't automatically execute Model Context Protocol
(MCP) tool calls in Gemini Live API sessions. To use MCP tools in a
Gemini Live API session, you can use one of the following approaches:

- **Manage MCP tool calls directly** : If you use the Google Gen AI SDK directly, your application must manage the MCP client session, execute tool calls when requested by the model, and return the results using `send_tool_response`.
- **Use the [Agent Development Kit (ADK)](https://adk.dev/)** : Use the `McpToolset` class in ADK, which provides built-in MCP integration for Gemini Live API. ADK automatically discovers tools from your MCP server, converts their schemas into function declarations, runs MCP tool calls during the bidirectional stream, and sends the tool responses back to the model.

#### Use MCP tools with ADK

You can use the ADK to connect MCP tools to Gemini Live API
using the following steps:

1. Install the `google-adk` package with the `mcp` extra:

   ```bash
   pip install "google-adk[mcp]"
   ```
2. Create a script named `live_mcp_example.py` that connects `McpToolset` to
   your MCP service using `StreamableHTTPConnectionParams` and streams events
   using `runner.run_live()`.

   Replace the following:
   - `PROJECT_ID`: your Google Cloud project ID.
   - `MCP_SERVER_URL`: your Streamable HTTP MCP server URL (for example, `http://127.0.0.1:8765/mcp`).

   ```python
   import asyncio
   import contextlib
   import os

   from google.adk.agents import Agent, LiveRequestQueue, RunConfig
   from google.adk.agents.run_config import StreamingMode
   from google.adk.runners import InMemoryRunner
   from google.adk.tools.mcp_tool import McpToolset, StreamableHTTPConnectionParams
   from google.genai import types

   os.environ.setdefault("GOOGLE_GENAI_USE_ENTERPRISE", "TRUE")
   os.environ.setdefault("GOOGLE_CLOUD_PROJECT", "PROJECT_ID")
   os.environ.setdefault("GOOGLE_CLOUD_LOCATION", "us-central1")

   async def main() -> None:
     # 1. Connect to the Streamable HTTP MCP service
     mcp_toolset = McpToolset(
         connection_params=StreamableHTTPConnectionParams(
             url="MCP_SERVER_URL",
         )
     )

     # 2. Create the ADK Agent with a Gemini Live model and the MCP toolset
     agent = Agent(
         name="mcp_live_agent",
         model="gemini-3.8-live",
         instruction="You are a helpful assistant. Use the provided MCP tools when needed.",
         tools=[mcp_toolset],
     )

     runner = InMemoryRunner(agent=agent, app_name="mcp_live_demo")
     session = await runner.session_service.create_session(
         app_name="mcp_live_demo",
         user_id="user_1",
     )

     # 3. Configure bidirectional streaming and send a prompt
     live_request_queue = LiveRequestQueue()
     run_config = RunConfig(
         streaming_mode=StreamingMode.BIDI,
         response_modalities=[types.Modality.AUDIO],
         output_audio_transcription=types.AudioTranscriptionConfig(),
     )

     live_request_queue.send_content(
         types.Content(
             role="user",
             parts=[types.Part.from_text(text="What is the weather in Paris?")],
         )
     )

     # 4. Stream Live events (ADK automatically executes MCP tools)
     awaiting_post_tool_response = False
     saw_function_response = False
     has_model_output = False

     try:
       async with contextlib.aclosing(
           runner.run_live(
               user_id="user_1",
               session_id=session.id,
               live_request_queue=live_request_queue,
               run_config=run_config,
           )
       ) as live_events:
         async for event in live_events:
           for call in event.get_function_calls():
             print(f"[MCP Tool Call] {call.name}({call.args})")
             awaiting_post_tool_response = True
             saw_function_response = False
             has_model_output = False

           for resp in event.get_function_responses():
             print(f"[MCP Tool Response] {resp.name} -> {resp.response}")
             saw_function_response = True

           if event.output_transcription and event.output_transcription.text:
             if not awaiting_post_tool_response or saw_function_response:
               has_model_output = True
               awaiting_post_tool_response = False
             if event.output_transcription.finished:
               print(f"[Transcript] {event.output_transcription.text}")

           # Wait for the final response turn after tool execution completes
           if event.turn_complete:
             if awaiting_post_tool_response or not has_model_output:
               continue
             live_request_queue.close()
             break
     finally:
       await mcp_toolset.close()
       await runner.close()

   if __name__ == "__main__":
     asyncio.run(main())
   ```

### Grounding with Google Search

Use [Grounding with Google Search](https://docs.cloud.google.com/gemini-enterprise-agent-platform/models/grounding/grounding-with-google-search.md.txt) when you want
the model to provide more accurate and factual responses by anchoring them to
verifiable sources of information. Use this for tasks like searching the web.

Unlike function calling, the server-side integration handles the retrieval of
information automatically.

To enable Grounding with Google Search, include `google_search` in the
`tools` list in the setup message:

### Python

```python
import asyncio

from google import genai
from google.genai.types import (
    Content,
    LiveConnectConfig,
    Part,
)

# Initialize the client.
client = genai.Client(
    enterprise=True,
    project="GOOGLE_CLOUD_PROJECT",  # Replace with your project ID
    location="LOCATION",  # Replace with your location
)

MODEL_ID = "gemini-3.8-live"


async def main():
    config = LiveConnectConfig(
        response_modalities=["AUDIO"],
        tools=[{"google_search": {}}],
    )

    async with client.aio.live.connect(
        model=MODEL_ID,
        config=config,
    ) as session:
        text_input = "What is the current weather in Toronto, Canada?"
        print(f"Input: {text_input}")

        await session.send_realtime_input(text=text_input)

        async for message in session.receive():
            # Consume the messages from the model.
            # In native audio, the model response is in audio format.
            pass


if __name__ == "__main__":
    asyncio.run(main())
  
```

## Configure Affective Dialog

Affective dialog configuration is only required for
`gemini-live-2.5-flash-native-audio`. This section doesn't
apply to `gemini-3.8-live`.

> [!IMPORTANT]
> **Important:** Affective Dialog can produce unexpected results.

When **Affective Dialog** is enabled, the model attempts to
understand and respond based on the tone of voice and emotional expressions of
the user.

To enable Affective Dialog, set `enable_affective_dialog` to
`true` in the setup message:

### Python

```python
config = LiveConnectConfig(
    response_modalities=["AUDIO"],
    enable_affective_dialog=True,
)
  
```

## Configure Proactive Audio

Proactive audio configuration is only required for
`gemini-live-2.5-flash-native-audio`. Proactive audio is
included in `gemini-3.8-live`.

Proactive Audio lets you control when the model responds. For
example, you can ask Gemini to only respond when prompted or when
specific topics are discussed. To see a video demonstration of
Proactive Audio, see
[Gemini LiveAPI Native Audio Preview](https://youtu.be/ZsB33Tr-P3c).

To enable Proactive Audio, configure the `proactivity` field
in the setup message and set `proactive_audio` to `true`:

### Python

```python
config = LiveConnectConfig(
    response_modalities=["AUDIO"],
    proactivity=ProactivityConfig(proactive_audio=True),
)
  
```

**Example conversation**

The following is a sample of what a conversation with Gemini about
cooking might look like:

```text
Prompt: "You are an AI assistant in Italian cooking; only chime in when the topic is about Italian cooking."

Speaker A: "I really love cooking!" (No response from Gemini.)

Speaker B: "Oh yes, me too! My favorite is French cuisine." (No response from
Gemini.)

Speaker A: "I really like Italian food; do you know how to make a pizza?"

(Italian cooking topic triggers response from Gemini.)
Gemini Live API: "I'd be happy to help! Here's a recipe for a pizza."
```

### Common use cases

When using Proactive Audio, Gemini performs as follows:

- **Responds with minimal latency**: Gemini responds after the user is done speaking, reducing interruptions and helping Gemini not lose context if an interruption happens.
- **Avoids interruptions**: Proactive Audio helps Gemini avoid interruptions from background noise or external chatter, and prevents Gemini from responding if external chatter is introduced during a conversation.
- **Handles interruptions** : If the user needs to interrupt during a response from Gemini, Proactive Audio makes it easier for Gemini to appropriately back-channel (meaning appropriate interruptions are handled), rather than if a user uses filler words such as *umm* or *uhh*.
- **Co-listens to audio**: Gemini can co-listen to an audio file that's not the speaker's voice and subsequently answer questions about that audio file later in the conversation.

### Billing

While Gemini is listening to a conversation, input audio tokens are
charged.

For output audio tokens, you're only charged when Gemini responds. If
Gemini does not respond or stays silent, there is no charge to
your output audio tokens.

For more information, see [Agent Platform pricing](https://cloud.google.com/gemini-enterprise-agent-platform/generative-ai/pricing).

## What's next

For more information on using Gemini Live API, see:

- [Gemini Live API overview](https://docs.cloud.google.com/gemini-enterprise-agent-platform/models/live-api.md.txt)
- [Gemini Live API reference guide](https://docs.cloud.google.com/gemini-enterprise-agent-platform/reference/models/live-api.md.txt)
- [Start and manage live sessions](https://docs.cloud.google.com/gemini-enterprise-agent-platform/models/live-api/start-manage-session.md.txt)