Upscale videos

You can use Gemini Omni 1.2 Flash to upscale videos from lower resolutions (360p or 720p) to higher resolutions (720p, 1080p, or 4K) using the Agent Platform API. Standalone video upscaling lets you iterate on prompts at lower resolutions and upscale only the final selected video, either from the preceding turn or from a previously generated video stored in Cloud Storage.

The following models support upscaling videos:

Click to expand supported models

Supported resolutions and specifications

Standalone video upscaling supports the following resolution transitions and input constraints:

  • 360p input video: Upscale to 720p or 1080p. Upscaling directly from 360p to 4k in a single request is not supported.
  • 720p input video: Upscale to 1080p or 4k. Upscaling from 1080p to 4k is not supported.
  • Aspect ratios: 16:9 (landscape) or 9:16 (portrait).
  • Frame rate: 24 frames per second (preserved in the output video).
  • Duration: Up to 15 seconds.
  • Token usage and pricing: Standalone video upscaling bypasses text reasoning and always incurs zero input tokens (total_input_tokens: 0). Output video tokens are billed at the same output video token price as all other Omni 1.2 Flash tasks, based on the source and target resolution pair (2,896 output video tokens per second for 360p to 720p and 720p to 1080p, 5,792 output video tokens per second for 360p to 1080p, and 11,584 output video tokens per second for 720p to 4k).

Request requirements

When sending a standalone video upscaling request, make sure your request meets the following requirements:

  • Specify task: "upscale": You must set generation_config.video_config.task to "upscale". If you omit task: "upscale", the API interprets the request as a video editing ("edit") operation.
  • Specify a target resolution: You must specify the target resolution ("720p", "1080p", or "4k") in response_format. Omitting the target resolution returns an error.
  • Provide a video source: Provide either an input video in input ("type": "video") or set previous_interaction_id to upscale a video generated in the immediately preceding stateful turn. In multi-turn interactions, only the immediately preceding turn is considered when locating the video to upscale.
  • Optional parameters: You can specify generation_config.seed for reproducible output and attach custom metadata labels (labels) to organize requests and track billing costs.

Try Gemini Omni in a Colab

Before you begin

  1. Sign in to your Google Cloud account. If you're new to Google Cloud, create an account to evaluate how our products perform in real-world scenarios. New customers also get $300 in free credits to run, test, and deploy workloads.
  2. In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

    Roles required to select or create a project

    • Select a project: Selecting a project doesn't require a specific IAM role—you can select any project that you've been granted a role on.
    • Create a project: To create a project, you need the Project Creator role (roles/resourcemanager.projectCreator), which contains the resourcemanager.projects.create permission. Learn how to grant roles.

    Go to project selector

  3. Enable the Agent Platform API, if it is not already enabled.

    Roles required to enable APIs

    To enable APIs, you need the serviceusage.services.enable permission. If you created the project, then you likely already have this permission through the Owner role (roles/owner). Otherwise, you can get this permission through the Service Usage Admin role (roles/serviceusage.serviceUsageAdmin). Learn how to grant roles.

    Enable the API

  4. In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

    Roles required to select or create a project

    • Select a project: Selecting a project doesn't require a specific IAM role—you can select any project that you've been granted a role on.
    • Create a project: To create a project, you need the Project Creator role (roles/resourcemanager.projectCreator), which contains the resourcemanager.projects.create permission. Learn how to grant roles.

    Go to project selector

  5. Enable the Agent Platform API, if it is not already enabled.

    Roles required to enable APIs

    To enable APIs, you need the serviceusage.services.enable permission. If you created the project, then you likely already have this permission through the Owner role (roles/owner). Otherwise, you can get this permission through the Service Usage Admin role (roles/serviceusage.serviceUsageAdmin). Learn how to grant roles.

    Enable the API

  6. Set up authentication for your environment.

    Select the tab for how you plan to use the samples on this page:

    Console

    When you use the Google Cloud console to access Google Cloud services and APIs, you don't need to set up authentication.

    REST

    To use the REST API samples on this page in a local development environment, you use the credentials you provide to the gcloud CLI.

      Install the Google Cloud CLI.

      If you're using an external identity provider (IdP), you must first sign in to the gcloud CLI with your federated identity.

    For more information, see Authenticate for using REST in the Google Cloud authentication documentation.

Upscale videos using Omni 1.2 Flash

To upscale videos using Omni 1.2 Flash, select one of the following interaction modes:

REST

Video upscaling can take over a minute to complete. You can choose from several interaction modes depending on your workflow:

  • Synchronous: stateless (uploaded video): Set store to false to upscale a 360p or 720p video from Cloud Storage in a single request without persisting interaction state on the server.
  • Synchronous: stateful (previous turn): Reference the previous_interaction_id from the immediately preceding stateful turn to upscale a previously generated 360p or 720p video.
  • Synchronous: stateful streaming: Set both store and stream to true to stream the interaction status updates and output video.
  • Asynchronous: Set background to true to run video upscaling in the background and retrieve the result later using the interaction ID.

Stored and asynchronous interactions are retained for up to 7 days.

For more information about using Gemini Omni models with the Interactions API, see Interactions API.

Synchronous: stateless

Before using any of the request data, make the following replacements:

  • PROJECT_ID: A string representing your Google Cloud project ID.
  • MODEL_ID: A string representing the model ID to use. The following is the accepted value:
    • "gemini-omni-1.2-flash"
  • CLOUD_STORAGE_VIDEO_URI: A string representing the Cloud Storage URI that contains the input video (360p or 720p, up to 15 seconds). For example: "gs://video-bucket/input/video.mp4".
  • VIDEO_MIME: A string representing a video MIME type. The following are accepted values:
    • video/3gpp
    • video/mp4
    • video/mpeg
    • video/quicktime
    • video/webm
    • video/x-flv
    • video/x-ms-wmv
  • CLOUD_STORAGE_OUTPUT_URI: Optional: A string representing the Cloud Storage bucket to store the upscaled video. If not provided, video bytes are returned in the response. For example: "gs://video-bucket/output/".
  • OUTPUT_RESOLUTION: Required: A string representing the target output resolution. Supported values depend on the input video resolution:
    • For 360p input videos: "720p" or "1080p"
    • For 720p input videos: "1080p" or "4k"

HTTP method and URL:

POST https://aiplatform.googleapis.com/v1beta1/projects/PROJECT_ID/locations/global/interactions

Request JSON body:

{
  "model": "MODEL_ID",
  "store": false,
  "input": [
    {
      "type": "video",
      "uri": "CLOUD_STORAGE_VIDEO_URI",
      "mime_type": "VIDEO_MIME"
    }
  ],
  "response_format": [
    {
      "type": "video",
      "delivery": "uri",
      "gcs_uri": "CLOUD_STORAGE_OUTPUT_URI",
      "resolution": "OUTPUT_RESOLUTION"
    }
  ],
  "generation_config": {
    "video_config": {
      "task": "upscale"
    }
  }
}

To send your request, choose one of these options:

curl

Save the request body in a file named request.json, and execute the following command:

curl -X POST \
-H "Authorization: Bearer TOKEN" \
-H "Content-Type: application/json; charset=utf-8" \
-d @request.json \
"https://aiplatform.googleapis.com/v1beta1/projects/PROJECT_ID/locations/global/interactions"

PowerShell

Save the request body in a file named request.json, and execute the following command:

$headers = @{ "Authorization" = "Bearer TOKEN" }

Invoke-WebRequest `
-Method POST `
-Headers $headers `
-ContentType: "application/json; charset=utf-8" `
-InFile request.json `
-Uri "https://aiplatform.googleapis.com/v1beta1/projects/PROJECT_ID/locations/global/interactions" | Select-Object -Expand Content
The response contains an interaction with the upscaled output video and zero input tokens.
{
  "id": "INTERACTION_ID",
  "model": "gemini-omni-1.2-flash",
  "status": "completed",
  "usage": {
    "total_tokens": 28960,
    "total_input_tokens": 0,
    "output_tokens_by_modality": [
      {
        "modality": "video",
        "tokens": 28960
      }
    ],
    "total_output_tokens": 28960
  },
  "steps": [
    { 
      "type": "model_output",
      "content": [
        {
          "type": "video",
          "uri": "gs://some/output_path/123.mp4",
          "mime_type": "video/mp4" 
        }
      ]
    }
  ],
  "object": "interaction",
  "role": "model",
  "created": "2026-11-05T02:17:56Z",
  "updated": "2026-11-05T02:17:56Z"
}

Synchronous: stateful

Before using any of the request data, make the following replacements:

  • PROJECT_ID: A string representing your Google Cloud project ID.
  • MODEL_ID: A string representing the model ID to use. The following is the accepted value:
    • "gemini-omni-1.2-flash"
  • PREVIOUS_INTERACTION_ID: The interaction ID of the immediately preceding stateful turn that generated a 360p or 720p video (up to 15 seconds).
  • CLOUD_STORAGE_OUTPUT_URI: Optional: A string representing the Cloud Storage bucket to store the upscaled video. If not provided, video bytes are returned in the response. For example: "gs://video-bucket/output/".
  • OUTPUT_RESOLUTION: Required: A string representing the target output resolution. Supported values depend on the video resolution from the previous turn:
    • For 360p videos: "720p" or "1080p"
    • For 720p videos: "1080p" or "4k"

HTTP method and URL:

POST https://aiplatform.googleapis.com/v1beta1/projects/PROJECT_ID/locations/global/interactions

Request JSON body:

{
  "model": "MODEL_ID",
  "previous_interaction_id": "PREVIOUS_INTERACTION_ID",
  "background": false,
  "store": true,
  "stream": false,
  "input": [
    {
      "type": "user_input",
      "content": [
        {
          "type": "text",
          "text": "Upsample video."
        }
      ]
    }
  ],
  "response_format": [
    {
      "type": "video",
      "delivery": "uri",
      "gcs_uri": "CLOUD_STORAGE_OUTPUT_URI",
      "resolution": "OUTPUT_RESOLUTION"
    }
  ],
  "generation_config": {
    "video_config": {
      "task": "upscale"
    }
  }
}

To send your request, choose one of these options:

curl

Save the request body in a file named request.json, and execute the following command:

curl -X POST \
-H "Authorization: Bearer TOKEN" \
-H "Content-Type: application/json; charset=utf-8" \
-d @request.json \
"https://aiplatform.googleapis.com/v1beta1/projects/PROJECT_ID/locations/global/interactions"

PowerShell

Save the request body in a file named request.json, and execute the following command:

$headers = @{ "Authorization" = "Bearer TOKEN" }

Invoke-WebRequest `
-Method POST `
-Headers $headers `
-ContentType: "application/json; charset=utf-8" `
-InFile request.json `
-Uri "https://aiplatform.googleapis.com/v1beta1/projects/PROJECT_ID/locations/global/interactions" | Select-Object -Expand Content
The response contains an interaction with the upscaled output video and zero input tokens.
{
  "id": "INTERACTION_ID",
  "previous_interaction_id": "PREVIOUS_INTERACTION_ID",
  "model": "gemini-omni-1.2-flash",
  "status": "completed",
  "usage": {
    "total_tokens": 28960,
    "total_input_tokens": 0,
    "output_tokens_by_modality": [
      {
        "modality": "video",
        "tokens": 28960
      }
    ],
    "total_output_tokens": 28960
  },
  "steps": [
    { 
      "type": "model_output",
      "content": [
        {
          "type": "video",
          "uri": "gs://some/output_path/123.mp4",
          "mime_type": "video/mp4" 
        }
      ]
    }
  ],
  "object": "interaction",
  "role": "model",
  "created": "2026-11-05T02:17:56Z",
  "updated": "2026-11-05T02:17:56Z"
}
Use the INTERACTION_ID to get the upscaled video:

Before using any of the request data, make the following replacements:

  • PROJECT_ID: A string representing your Google Cloud project ID.
  • INTERACTION_ID: The interaction ID from the asynchronous request.

HTTP method and URL:

GET https://aiplatform.googleapis.com/v1beta1/projects/PROJECT_ID/locations/global/interactions/INTERACTION_ID

To send your request, choose one of these options:

curl

Execute the following command:

curl -X GET \
-H "Authorization: Bearer TOKEN" \
"https://aiplatform.googleapis.com/v1beta1/projects/PROJECT_ID/locations/global/interactions/INTERACTION_ID"

PowerShell

Execute the following command:

$headers = @{ "Authorization" = "Bearer TOKEN" }

Invoke-WebRequest `
-Method GET `
-Headers $headers `
-Uri "https://aiplatform.googleapis.com/v1beta1/projects/PROJECT_ID/locations/global/interactions/INTERACTION_ID" | Select-Object -Expand Content
The response is in a format similar to the following:
{
  "id":"INTERACTION_ID",
  "model":"gemini-omni-1.2-flash",
  "status":"completed",
  "usage":{
    "total_tokens":479,
    "total_input_tokens":26,
    "input_tokens_by_modality":[
      {
        "modality":"text",
        "tokens":26
      },
      {
        "modality":"image",
        "tokens":124
      }
    ],
    "output_tokens_by_modality": [
      {
        "modality": "video",
        "tokens": 28832
      }
    ],
    "total_output_tokens":28832,
    "total_thought_tokens":453
  },
  "steps":[
    {
      "type": "user_input",
      "content": [
        {
          "type": "text",
          "text": "5 second, 9:16 video. Use the image as the first frame."
        },
        {
          "type": "image",
          "uri": "gs://some/path",
          "mime_type": "image/png"
        }
      ]
    },
    {
      "type":"thought",
      "summary":[
        {
          "type":"text",
          "text":"MODEL THOUGHTS"
        }
      ]
    },
    { 
      "type":"model_output",
      "content":[
        {
          "type":"video",
          "data":"VIDEO DATA",
          "mime_type":"video/mp4" 
        }
      ]
    }
  ],
  "object":"interaction",
  "role":"model",
  "created":"2026-05-29T02:17:56Z",
  "updated":"2026-05-29T02:17:56Z"
}

Synchronous: stateful streaming

Before using any of the request data, make the following replacements:

  • PROJECT_ID: A string representing your Google Cloud project ID.
  • MODEL_ID: A string representing the model ID to use. The following is the accepted value:
    • "gemini-omni-1.2-flash"
  • CLOUD_STORAGE_VIDEO_URI: A string representing the Cloud Storage URI that contains the input video (360p or 720p, up to 15 seconds). For example: "gs://video-bucket/input/video.mp4".
  • VIDEO_MIME: A string representing a video MIME type. The following are accepted values:
    • video/3gpp
    • video/mp4
    • video/mpeg
    • video/quicktime
    • video/webm
    • video/x-flv
    • video/x-ms-wmv
  • CLOUD_STORAGE_OUTPUT_URI: Optional: A string representing the Cloud Storage bucket to store the upscaled video. If not provided, video bytes are returned in the response. For example: "gs://video-bucket/output/".
  • OUTPUT_RESOLUTION: Required: A string representing the target output resolution. Supported values depend on the input video resolution:
    • For 360p input videos: "720p" or "1080p"
    • For 720p input videos: "1080p" or "4k"

HTTP method and URL:

POST https://aiplatform.googleapis.com/v1beta1/projects/PROJECT_ID/locations/global/interactions

Request JSON body:

{
  "model": "MODEL_ID",
  "background": false,
  "store": true,
  "stream": true,
  "input": [
    {
      "type": "user_input",
      "content": [
        {
          "type": "video",
          "uri": "CLOUD_STORAGE_VIDEO_URI",
          "mime_type": "VIDEO_MIME"
        }
      ]
    }
  ],
  "response_format": [
    {
      "type": "video",
      "delivery": "uri",
      "gcs_uri": "CLOUD_STORAGE_OUTPUT_URI",
      "resolution": "OUTPUT_RESOLUTION"
    }
  ],
  "generation_config": {
    "video_config": {
      "task": "upscale"
    }
  }
}

To send your request, choose one of these options:

curl

Save the request body in a file named request.json, and execute the following command:

curl -X POST \
-H "Authorization: Bearer TOKEN" \
-H "Content-Type: application/json; charset=utf-8" \
-d @request.json \
"https://aiplatform.googleapis.com/v1beta1/projects/PROJECT_ID/locations/global/interactions"

PowerShell

Save the request body in a file named request.json, and execute the following command:

$headers = @{ "Authorization" = "Bearer TOKEN" }

Invoke-WebRequest `
-Method POST `
-Headers $headers `
-ContentType: "application/json; charset=utf-8" `
-InFile request.json `
-Uri "https://aiplatform.googleapis.com/v1beta1/projects/PROJECT_ID/locations/global/interactions" | Select-Object -Expand Content
The response is a stream of Server-Sent Events (SSE) containing the upscaled output video.
event: interaction.created
data: {"interaction":{"id":"INTERACTION_ID","status":"in_progress","object":"interaction","model":"gemini-omni-1.2-flash"},"event_type":"interaction.created"}

event: interaction.status_update
data: {"interaction_id":"INTERACTION_ID","status":"in_progress","event_type":"interaction.status_update"}

event: step.start
data: {"index":0,"step":{"type":"model_output"},"event_type":"step.start"}

event: step.delta
data: {"index":0,"delta":{"mime_type":"video/mp4","uri":"gs://some/output_path/123.mp4","type":"video"},"event_id":"EVENT_ID","event_type":"step.delta"}

event: step.stop
data: {"index":0,"event_type":"step.stop"}

event: interaction.completed
data: {"interaction":{"id":"INTERACTION_ID","status":"completed","usage":{"total_tokens":28960,"total_input_tokens":0,"total_output_tokens":28960,"output_tokens_by_modality":[{"modality":"video","tokens":28960}]},"created":"2026-11-05T21:26:20Z","updated":"2026-11-05T21:26:20Z","event_id":"EVENT_ID","object":"interaction","model":"gemini-omni-1.2-flash"},"event_id":"EVENT_ID","event_type":"interaction.completed"}

event: done
data: [DONE]
Use the INTERACTION_ID with `?stream=true` to stream the stored interaction and upscaled video:

Before using any of the request data, make the following replacements:

  • PROJECT_ID: A string representing your Google Cloud project ID.
  • INTERACTION_ID: The interaction ID from the request.

HTTP method and URL:

GET https://aiplatform.googleapis.com/v1beta1/projects/PROJECT_ID/locations/global/interactions/INTERACTION_ID?stream=true

To send your request, choose one of these options:

curl

Execute the following command:

curl -X GET \
-H "Authorization: Bearer TOKEN" \
"https://aiplatform.googleapis.com/v1beta1/projects/PROJECT_ID/locations/global/interactions/INTERACTION_ID?stream=true"

PowerShell

Execute the following command:

$headers = @{ "Authorization" = "Bearer TOKEN" }

Invoke-WebRequest `
-Method GET `
-Headers $headers `
-Uri "https://aiplatform.googleapis.com/v1beta1/projects/PROJECT_ID/locations/global/interactions/INTERACTION_ID?stream=true" | Select-Object -Expand Content
The response is a stream of Server-Sent Events (SSE) in a format similar to the following:
event: interaction.created
data: {"interaction":{"id":"INTERACTION_ID","status":"in_progress","object":"interaction","model":"gemini-omni-1.2-flash"},"event_type":"interaction.created"}

event: interaction.status_update
data: {"interaction_id":"INTERACTION_ID","status":"in_progress","event_type":"interaction.status_update"}

event: step.start
data: {"index":0,"step":{"type":"thought"},"event_type":"step.start"}

event: step.delta
data: {"index":0,"delta":{"content":{"text":"MODEL THOUGHTS","type":"text"},"type":"thought_summary"},"event_id":"EVENT_ID","event_type":"step.delta"}

event: step.delta
data: {"index":0,"delta":{"content":{"text":"MODEL THOUGHTS","type":"text"},"type":"thought_summary"},"event_id":"EVENT_ID","event_type":"step.delta"}

event: step.delta
data: {"index":0,"delta":{"signature":"...","type":"thought_signature"},"event_type":"step.delta"}

event: step.stop
data: {"index":0,"event_type":"step.stop"}

event: step.start
data: {"index":1,"step":{"type":"model_output"},"event_type":"step.start"}

event: step.delta
data: {"index":1,"delta":{"mime_type":"video/mp4","uri":"gs://some/output_path/123.mp4","type":"video"},"event_id":"EVENT_ID","event_type":"step.delta"}

event: step.stop
data: {"index":1,"event_type":"step.stop"}

event: interaction.completed
data: {"interaction":{"id":"INTERACTION_ID","status":"completed","usage":{"total_tokens":17664,"total_input_tokens":21,"input_tokens_by_modality":[{"modality":"text","tokens":21}],"total_output_tokens":17376,"output_tokens_by_modality":[{"modality":"video","tokens":17376}],"total_thought_tokens":267},"created":"2026-08-12T21:33:08Z","updated":"2026-08-12T21:33:08Z","event_id":"EVENT_ID","object":"interaction","model":"gemini-omni-1.2-flash"},"event_id":"EVENT_ID","event_type":"interaction.completed"}

event: done
data: [DONE]

Asynchronous

Before using any of the request data, make the following replacements:

  • PROJECT_ID: A string representing your Google Cloud project ID.
  • MODEL_ID: A string representing the model ID to use. The following is the accepted value:
    • "gemini-omni-1.2-flash"
  • CLOUD_STORAGE_VIDEO_URI: A string representing the Cloud Storage URI that contains the input video (360p or 720p, up to 15 seconds). For example: "gs://video-bucket/input/video.mp4".
  • VIDEO_MIME: A string representing a video MIME type. The following are accepted values:
    • video/3gpp
    • video/mp4
    • video/mpeg
    • video/quicktime
    • video/webm
    • video/x-flv
    • video/x-ms-wmv
  • CLOUD_STORAGE_OUTPUT_URI: Optional: A string representing the Cloud Storage bucket to store the upscaled video. If not provided, video bytes are returned in the response. For example: "gs://video-bucket/output/".
  • OUTPUT_RESOLUTION: Required: A string representing the target output resolution. Supported values depend on the input video resolution:
    • For 360p input videos: "720p" or "1080p"
    • For 720p input videos: "1080p" or "4k"

HTTP method and URL:

POST https://aiplatform.googleapis.com/v1beta1/projects/PROJECT_ID/locations/global/interactions

Request JSON body:

{
  "model": "MODEL_ID",
  "background": true,
  "input": [
    {
      "type": "video",
      "uri": "CLOUD_STORAGE_VIDEO_URI",
      "mime_type": "VIDEO_MIME"
    }
  ],
  "response_format": [
    {
      "type": "video",
      "delivery": "uri",
      "gcs_uri": "CLOUD_STORAGE_OUTPUT_URI",
      "resolution": "OUTPUT_RESOLUTION"
    }
  ],
  "generation_config": {
    "video_config": {
      "task": "upscale"
    }
  }
}

To send your request, choose one of these options:

curl

Save the request body in a file named request.json, and execute the following command:

curl -X POST \
-H "Authorization: Bearer TOKEN" \
-H "Content-Type: application/json; charset=utf-8" \
-d @request.json \
"https://aiplatform.googleapis.com/v1beta1/projects/PROJECT_ID/locations/global/interactions"

PowerShell

Save the request body in a file named request.json, and execute the following command:

$headers = @{ "Authorization" = "Bearer TOKEN" }

Invoke-WebRequest `
-Method POST `
-Headers $headers `
-ContentType: "application/json; charset=utf-8" `
-InFile request.json `
-Uri "https://aiplatform.googleapis.com/v1beta1/projects/PROJECT_ID/locations/global/interactions" | Select-Object -Expand Content
The response contains an interaction ID, which you use to retrieve the upscaled video.
{
  "id": "INTERACTION_ID",
  "status": "in_progress",
  "object": "interaction"
}
Later, use the INTERACTION_ID to get the upscaled video:

Before using any of the request data, make the following replacements:

  • PROJECT_ID: A string representing your Google Cloud project ID.
  • INTERACTION_ID: The interaction ID from the asynchronous request.

HTTP method and URL:

GET https://aiplatform.googleapis.com/v1beta1/projects/PROJECT_ID/locations/global/interactions/INTERACTION_ID

To send your request, choose one of these options:

curl

Execute the following command:

curl -X GET \
-H "Authorization: Bearer TOKEN" \
"https://aiplatform.googleapis.com/v1beta1/projects/PROJECT_ID/locations/global/interactions/INTERACTION_ID"

PowerShell

Execute the following command:

$headers = @{ "Authorization" = "Bearer TOKEN" }

Invoke-WebRequest `
-Method GET `
-Headers $headers `
-Uri "https://aiplatform.googleapis.com/v1beta1/projects/PROJECT_ID/locations/global/interactions/INTERACTION_ID" | Select-Object -Expand Content
The response is in a format similar to the following:
{
  "id":"INTERACTION_ID",
  "model":"gemini-omni-1.2-flash",
  "status":"completed",
  "usage":{
    "total_tokens":479,
    "total_input_tokens":26,
    "input_tokens_by_modality":[
      {
        "modality":"text",
        "tokens":26
      },
      {
        "modality":"image",
        "tokens":124
      }
    ],
    "output_tokens_by_modality": [
      {
        "modality": "video",
        "tokens": 28832
      }
    ],
    "total_output_tokens":28832,
    "total_thought_tokens":453
  },
  "steps":[
    {
      "type": "user_input",
      "content": [
        {
          "type": "text",
          "text": "5 second, 9:16 video. Use the image as the first frame."
        },
        {
          "type": "image",
          "uri": "gs://some/path",
          "mime_type": "image/png"
        }
      ]
    },
    {
      "type":"thought",
      "summary":[
        {
          "type":"text",
          "text":"MODEL THOUGHTS"
        }
      ]
    },
    { 
      "type":"model_output",
      "content":[
        {
          "type":"video",
          "data":"VIDEO DATA",
          "mime_type":"video/mp4" 
        }
      ]
    }
  ],
  "object":"interaction",
  "role":"model",
  "created":"2026-05-29T02:17:56Z",
  "updated":"2026-05-29T02:17:56Z"
}

What's next