# Method: videos.annotate

- [HTTP request](https://docs.cloud.google.com/video-intelligence/docs/reference/rest/v1p1beta1/videos/annotate#body.HTTP_TEMPLATE)
- [Request body](https://docs.cloud.google.com/video-intelligence/docs/reference/rest/v1p1beta1/videos/annotate#body.request_body)
  - [JSON representation](https://docs.cloud.google.com/video-intelligence/docs/reference/rest/v1p1beta1/videos/annotate#body.request_body.SCHEMA_REPRESENTATION)
- [Response body](https://docs.cloud.google.com/video-intelligence/docs/reference/rest/v1p1beta1/videos/annotate#body.response_body)
  - [JSON representation](https://docs.cloud.google.com/video-intelligence/docs/reference/rest/v1p1beta1/videos/annotate#body.Operation.SCHEMA_REPRESENTATION)
- [Authorization Scopes](https://docs.cloud.google.com/video-intelligence/docs/reference/rest/v1p1beta1/videos/annotate#body.aspect)
- [Feature](https://docs.cloud.google.com/video-intelligence/docs/reference/rest/v1p1beta1/videos/annotate#Feature)
- [VideoContext](https://docs.cloud.google.com/video-intelligence/docs/reference/rest/v1p1beta1/videos/annotate#VideoContext)
  - [JSON representation](https://docs.cloud.google.com/video-intelligence/docs/reference/rest/v1p1beta1/videos/annotate#VideoContext.SCHEMA_REPRESENTATION)
- [VideoSegment](https://docs.cloud.google.com/video-intelligence/docs/reference/rest/v1p1beta1/videos/annotate#VideoSegment)
  - [JSON representation](https://docs.cloud.google.com/video-intelligence/docs/reference/rest/v1p1beta1/videos/annotate#VideoSegment.SCHEMA_REPRESENTATION)
- [LabelDetectionConfig](https://docs.cloud.google.com/video-intelligence/docs/reference/rest/v1p1beta1/videos/annotate#LabelDetectionConfig)
  - [JSON representation](https://docs.cloud.google.com/video-intelligence/docs/reference/rest/v1p1beta1/videos/annotate#LabelDetectionConfig.SCHEMA_REPRESENTATION)
- [LabelDetectionMode](https://docs.cloud.google.com/video-intelligence/docs/reference/rest/v1p1beta1/videos/annotate#LabelDetectionMode)
- [ShotChangeDetectionConfig](https://docs.cloud.google.com/video-intelligence/docs/reference/rest/v1p1beta1/videos/annotate#ShotChangeDetectionConfig)
  - [JSON representation](https://docs.cloud.google.com/video-intelligence/docs/reference/rest/v1p1beta1/videos/annotate#ShotChangeDetectionConfig.SCHEMA_REPRESENTATION)
- [ExplicitContentDetectionConfig](https://docs.cloud.google.com/video-intelligence/docs/reference/rest/v1p1beta1/videos/annotate#ExplicitContentDetectionConfig)
  - [JSON representation](https://docs.cloud.google.com/video-intelligence/docs/reference/rest/v1p1beta1/videos/annotate#ExplicitContentDetectionConfig.SCHEMA_REPRESENTATION)
- [SpeechTranscriptionConfig](https://docs.cloud.google.com/video-intelligence/docs/reference/rest/v1p1beta1/videos/annotate#SpeechTranscriptionConfig)
  - [JSON representation](https://docs.cloud.google.com/video-intelligence/docs/reference/rest/v1p1beta1/videos/annotate#SpeechTranscriptionConfig.SCHEMA_REPRESENTATION)
- [SpeechContext](https://docs.cloud.google.com/video-intelligence/docs/reference/rest/v1p1beta1/videos/annotate#SpeechContext)
  - [JSON representation](https://docs.cloud.google.com/video-intelligence/docs/reference/rest/v1p1beta1/videos/annotate#SpeechContext.SCHEMA_REPRESENTATION)
- [Status](https://docs.cloud.google.com/video-intelligence/docs/reference/rest/v1p1beta1/videos/annotate#Status)
  - [JSON representation](https://docs.cloud.google.com/video-intelligence/docs/reference/rest/v1p1beta1/videos/annotate#Status.SCHEMA_REPRESENTATION)
- [Try it!](https://docs.cloud.google.com/video-intelligence/docs/reference/rest/v1p1beta1/videos/annotate#try-it)

Performs asynchronous video annotation. Progress and results can be retrieved through the `google.longrunning.Operations` interface. `Operation.metadata` contains `AnnotateVideoProgress` (progress). `Operation.response` contains `AnnotateVideoResponse` (results).

### HTTP request

`POST https://videointelligence.googleapis.com/v1p1beta1/videos:annotate`

The URL uses [gRPC Transcoding](https://github.com/googleapis/googleapis/blob/master/google/api/http.proto) syntax.

### Request body

The request body contains data with the following structure:

| JSON representation ||
|---|---|
| ``` { "inputUri": string, "inputContent": string, "features": [ enum(`https://docs.cloud.google.com/video-intelligence/docs/reference/rest/v1p1beta1/videos/annotate#Feature`) ], "videoContext": { object(`https://docs.cloud.google.com/video-intelligence/docs/reference/rest/v1p1beta1/videos/annotate#VideoContext`) }, "outputUri": string, "locationId": string } ``` |

| Fields ||
|---|---|
| `inputUri` | `string` Input video location. Currently, only [Google Cloud Storage](https://cloud.google.com/storage/) URIs are supported, which must be specified in the following format: `gs://bucket-id/object-id` (other URI formats return `google.rpc.Code.INVALID_ARGUMENT`). For more information, see [Request URIs](https://docs.cloud.google.com/storage/docs/reference-uris). A video URI may include wildcards in `object-id`, and thus identify multiple videos. Supported wildcards: '\*' to match 0 or more characters; '?' to match 1 character. If unset, the input video should be embedded in the request as `inputContent`. If set, `inputContent` should be unset. |
| `inputContent` | `string (https://developers.google.com/discovery/v1/type-format format)` The video data bytes. If unset, the input video(s) should be specified via `inputUri`. If set, `inputUri` should be unset. A base64-encoded string. <br /> |
| `features[]` | ``enum(`https://docs.cloud.google.com/video-intelligence/docs/reference/rest/v1p1beta1/videos/annotate#Feature`)`` Requested video annotation features. |
| `videoContext` | ``object(`https://docs.cloud.google.com/video-intelligence/docs/reference/rest/v1p1beta1/videos/annotate#VideoContext`)`` Additional video context and/or feature-specific parameters. |
| `outputUri` | `string` Optional location where the output (in JSON format) should be stored. Currently, only [Google Cloud Storage](https://cloud.google.com/storage/) URIs are supported, which must be specified in the following format: `gs://bucket-id/object-id` (other URI formats return `google.rpc.Code.INVALID_ARGUMENT`). For more information, see [Request URIs](https://docs.cloud.google.com/storage/docs/reference-uris). |
| `locationId` | `string` Optional cloud region where annotation should take place. Supported cloud regions: `us-east1`, `us-west1`, `europe-west1`, `asia-east1`. If no region is specified, a region will be determined based on video file location. |

### Response body

If successful, the response body contains data with the following structure:
This resource represents a long-running operation that is the result of a network API call.

| JSON representation ||
|---|---|
| ``` { "name": string, "metadata": { "@type": string, field1: ..., ... }, "done": boolean, // Union field `result` can be only one of the following: "error": { object(`https://docs.cloud.google.com/video-intelligence/docs/reference/rest/v1p1beta1/videos/annotate#Status`) }, "response": { "@type": string, field1: ..., ... } // End of list of possible types for union field `result`. } ``` |

| Fields ||
|---|---|---|
| `name` | `string` The server-assigned name, which is only unique within the same service that originally returns it. If you use the default HTTP mapping, the `name` should have the format of `operations/some/unique/name`. |
| `metadata` | `object` Service-specific metadata associated with the operation. It typically contains progress information and common metadata such as create time. Some services might not provide such metadata. Any method that returns a long-running operation should document the metadata type, if any. An object containing fields of an arbitrary type. An additional field `"@type"` contains a URI identifying the type. Example: `{ "id": 1234, "@type": "types.example.com/standard/id" }`. <br /> |
| `done` | `boolean` If the value is `false`, it means the operation is still in progress. If `true`, the operation is completed, and either `error` or `response` is available. |
| Union field `result`. The operation result, which can be either an `error` or a valid `response`. If `done` == `false`, neither `error` nor `response` is set. If `done` == `true`, exactly one of `error` or `response` is set. `result` can be only one of the following: |||
| `error` | ``object(`https://docs.cloud.google.com/video-intelligence/docs/reference/rest/v1p1beta1/videos/annotate#Status`)`` The error result of the operation in case of failure or cancellation. |
| `response` | `object` The normal response of the operation in case of success. If the original method returns no data on success, such as `Delete`, the response is `google.protobuf.Empty`. If the original method is standard `Get`/`Create`/`Update`, the response should be the resource. For other methods, the response should have the type `XxxResponse`, where `Xxx` is the original method name. For example, if the original method name is `TakeSnapshot()`, the inferred response type is `TakeSnapshotResponse`. An object containing fields of an arbitrary type. An additional field `"@type"` contains a URI identifying the type. Example: `{ "id": 1234, "@type": "types.example.com/standard/id" }`. <br /> |

### Authorization Scopes

Requires the following OAuth scope:

- `https://www.googleapis.com/auth/cloud-platform`

For more information, see the [Authentication Overview](https://cloud.google.com/docs/authentication/).

## Feature

Video annotation feature.

| Enums ||
|---|---|
| `FEATURE_UNSPECIFIED` | Unspecified. |
| `LABEL_DETECTION` | Label detection. Detect objects, such as dog or flower. |
| `SHOT_CHANGE_DETECTION` | Shot change detection. |
| `EXPLICIT_CONTENT_DETECTION` | Explicit content detection. |
| `SPEECH_TRANSCRIPTION` | Speech transcription. |

## VideoContext

Video context and/or feature-specific parameters.

| JSON representation ||
|---|---|
| ``` { "segments": [ { object(`https://docs.cloud.google.com/video-intelligence/docs/reference/rest/v1p1beta1/videos/annotate#VideoSegment`) } ], "labelDetectionConfig": { object(`https://docs.cloud.google.com/video-intelligence/docs/reference/rest/v1p1beta1/videos/annotate#LabelDetectionConfig`) }, "shotChangeDetectionConfig": { object(`https://docs.cloud.google.com/video-intelligence/docs/reference/rest/v1p1beta1/videos/annotate#ShotChangeDetectionConfig`) }, "explicitContentDetectionConfig": { object(`https://docs.cloud.google.com/video-intelligence/docs/reference/rest/v1p1beta1/videos/annotate#ExplicitContentDetectionConfig`) }, "speechTranscriptionConfig": { object(`https://docs.cloud.google.com/video-intelligence/docs/reference/rest/v1p1beta1/videos/annotate#SpeechTranscriptionConfig`) } } ``` |

| Fields ||
|---|---|
| `segments[]` | ``object(`https://docs.cloud.google.com/video-intelligence/docs/reference/rest/v1p1beta1/videos/annotate#VideoSegment`)`` Video segments to annotate. The segments may overlap and are not required to be contiguous or span the whole video. If unspecified, each video is treated as a single segment. |
| `labelDetectionConfig` | ``object(`https://docs.cloud.google.com/video-intelligence/docs/reference/rest/v1p1beta1/videos/annotate#LabelDetectionConfig`)`` Config for LABEL_DETECTION. |
| `shotChangeDetectionConfig` | ``object(`https://docs.cloud.google.com/video-intelligence/docs/reference/rest/v1p1beta1/videos/annotate#ShotChangeDetectionConfig`)`` Config for SHOT_CHANGE_DETECTION. |
| `explicitContentDetectionConfig` | ``object(`https://docs.cloud.google.com/video-intelligence/docs/reference/rest/v1p1beta1/videos/annotate#ExplicitContentDetectionConfig`)`` Config for EXPLICIT_CONTENT_DETECTION. |
| `speechTranscriptionConfig` | ``object(`https://docs.cloud.google.com/video-intelligence/docs/reference/rest/v1p1beta1/videos/annotate#SpeechTranscriptionConfig`)`` Config for SPEECH_TRANSCRIPTION. |

## VideoSegment

Video segment.

| JSON representation ||
|---|---|
| ``` { "startTimeOffset": string, "endTimeOffset": string } ``` |

| Fields ||
|---|---|
| `startTimeOffset` | ``string (`https://developers.google.com/protocol-buffers/docs/reference/google.protobuf#google.protobuf.Duration` format)`` Time-offset, relative to the beginning of the video, corresponding to the start of the segment (inclusive). A duration in seconds with up to nine fractional digits, terminated by '`s`'. Example: `"3.5s"`. <br /> |
| `endTimeOffset` | ``string (`https://developers.google.com/protocol-buffers/docs/reference/google.protobuf#google.protobuf.Duration` format)`` Time-offset, relative to the beginning of the video, corresponding to the end of the segment (inclusive). A duration in seconds with up to nine fractional digits, terminated by '`s`'. Example: `"3.5s"`. <br /> |

## LabelDetectionConfig

Config for LABEL_DETECTION.

| JSON representation ||
|---|---|
| ``` { "labelDetectionMode": enum(`https://docs.cloud.google.com/video-intelligence/docs/reference/rest/v1p1beta1/videos/annotate#LabelDetectionMode`), "stationaryCamera": boolean, "model": string } ``` |

| Fields ||
|---|---|
| `labelDetectionMode` | ``enum(`https://docs.cloud.google.com/video-intelligence/docs/reference/rest/v1p1beta1/videos/annotate#LabelDetectionMode`)`` What labels should be detected with LABEL_DETECTION, in addition to video-level labels or segment-level labels. If unspecified, defaults to `SHOT_MODE`. |
| `stationaryCamera` | `boolean` Whether the video has been shot from a stationary (i.e. non-moving) camera. When set to true, might improve detection accuracy for moving objects. Should be used with `SHOT_AND_FRAME_MODE` enabled. |
| `model` | `string` Model to use for label detection. Supported values: "builtin/stable" (the default if unset) and "builtin/latest". |

## LabelDetectionMode

Label detection mode.

| Enums ||
|---|---|
| `LABEL_DETECTION_MODE_UNSPECIFIED` | Unspecified. |
| `SHOT_MODE` | Detect shot-level labels. |
| `FRAME_MODE` | Detect frame-level labels. |
| `SHOT_AND_FRAME_MODE` | Detect both shot-level and frame-level labels. |

## ShotChangeDetectionConfig

Config for SHOT_CHANGE_DETECTION.

| JSON representation ||
|---|---|
| ``` { "model": string } ``` |

| Fields ||
|---|---|
| `model` | `string` Model to use for shot change detection. Supported values: "builtin/stable" (the default if unset) and "builtin/latest". |

## ExplicitContentDetectionConfig

Config for EXPLICIT_CONTENT_DETECTION.

| JSON representation ||
|---|---|
| ``` { "model": string } ``` |

| Fields ||
|---|---|
| `model` | `string` Model to use for explicit content detection. Supported values: "builtin/stable" (the default if unset) and "builtin/latest". |

## SpeechTranscriptionConfig

Config for SPEECH_TRANSCRIPTION.

| JSON representation ||
|---|---|
| ``` { "languageCode": string, "maxAlternatives": number, "filterProfanity": boolean, "speechContexts": [ { object(`https://docs.cloud.google.com/video-intelligence/docs/reference/rest/v1p1beta1/videos/annotate#SpeechContext`) } ], "enableAutomaticPunctuation": boolean, "audioTracks": [ number ] } ``` |

| Fields ||
|---|---|
| `languageCode` | `string` *Required* The language of the supplied audio as a [BCP-47](https://www.rfc-editor.org/rfc/bcp/bcp47.txt) language tag. Example: "en-US". See [Language Support](https://cloud.google.com/speech/docs/languages) for a list of the currently supported language codes. |
| `maxAlternatives` | `number` *Optional* Maximum number of recognition hypotheses to be returned. Specifically, the maximum number of `SpeechRecognitionAlternative` messages within each `SpeechRecognitionResult`. The server may return fewer than `maxAlternatives`. Valid values are `0`-`30`. A value of `0` or `1` will return a maximum of one. If omitted, will return a maximum of one. |
| `filterProfanity` | `boolean` *Optional* If set to `true`, the server will attempt to filter out profanities, replacing all but the initial character in each filtered word with asterisks, e.g. "f\*\*\*". If set to `false` or omitted, profanities won't be filtered out. |
| `speechContexts[]` | ``object(`https://docs.cloud.google.com/video-intelligence/docs/reference/rest/v1p1beta1/videos/annotate#SpeechContext`)`` *Optional* A means to provide context to assist the speech recognition. |
| `enableAutomaticPunctuation` | `boolean` *Optional* If 'true', adds punctuation to recognition result hypotheses. This feature is only available in select languages. Setting this for requests in other languages has no effect at all. The default 'false' value does not add punctuation to result hypotheses. NOTE: "This is currently offered as an experimental service, complimentary to all users. In the future this may be exclusively available as a premium feature." |
| `audioTracks[]` | `number` *Optional* For file formats, such as MXF or MKV, supporting multiple audio tracks, specify up to two tracks. Default: track 0. |

## SpeechContext

Provides "hints" to the speech recognizer to favor specific words and phrases in the results.

| JSON representation ||
|---|---|
| ``` { "phrases": [ string ] } ``` |

| Fields ||
|---|---|
| `phrases[]` | `string` *Optional* A list of strings containing words and phrases "hints" so that the speech recognition is more likely to recognize them. This can be used to improve the accuracy for specific words and phrases, for example, if specific commands are typically spoken by the user. This can also be used to add additional words to the vocabulary of the recognizer. See [usage limits](https://cloud.google.com/speech/limits#content). |

## Status

The `Status` type defines a logical error model that is suitable for different programming environments, including REST APIs and RPC APIs. It is used by [gRPC](https://github.com/grpc). The error model is designed to be:

- Simple to use and understand for most users
- Flexible enough to meet unexpected needs

#### Overview

The `Status` message contains three pieces of data: error code, error message, and error details. The error code should be an enum value of `google.rpc.Code`, but it may accept additional error codes if needed. The error message should be a developer-facing English message that helps developers *understand* and *resolve* the error. If a localized user-facing error message is needed, put the localized message in the error details or localize it in the client. The optional error details may contain arbitrary information about the error. There is a predefined set of error detail types in the package `google.rpc` that can be used for common error conditions.

#### Language mapping

The `Status` message is the logical representation of the error model, but it is not necessarily the actual wire format. When the `Status` message is exposed in different client libraries and different wire protocols, it can be mapped differently. For example, it will likely be mapped to some exceptions in Java, but more likely mapped to some error codes in C.

#### Other uses

The error model and the `Status` message can be used in a variety of environments, either with or without APIs, to provide a consistent developer experience across different environments.

Example uses of this error model include:

- Partial errors. If a service needs to return partial errors to the client, it may embed the `Status` in the normal response to indicate the partial errors.

- Workflow errors. A typical workflow has multiple steps. Each step may have a `Status` message for error reporting.

- Batch operations. If a client uses batch request and batch response, the `Status` message should be used directly inside batch response, one for each error sub-response.

- Asynchronous operations. If an API call embeds asynchronous operation results in its response, the status of those operations should be represented directly using the `Status` message.

- Logging. If some API errors are stored in logs, the message `Status` could be used directly after any stripping needed for security/privacy reasons.

| JSON representation ||
|---|---|
| ``` { "code": number, "message": string, "details": [ { "@type": string, field1: ..., ... } ] } ``` |

| Fields ||
|---|---|
| `code` | `number` The status code, which should be an enum value of `google.rpc.Code`. |
| `message` | `string` A developer-facing error message, which should be in English. Any user-facing error message should be localized and sent in the `https://docs.cloud.google.com/video-intelligence/docs/reference/rest/v1p1beta1/videos/annotate#Status.FIELDS.details` field, or localized by the client. |
| `details[]` | `object` A list of messages that carry the error details. There is a common set of message types for APIs to use. An object containing fields of an arbitrary type. An additional field `"@type"` contains a URI identifying the type. Example: `{ "id": 1234, "@type": "types.example.com/standard/id" }`. <br /> |

## Try it!