The Video Intelligence API can identify entities shown in video footage
using the [LABEL_DETECTION](https://docs.cloud.google.com/video-intelligence/docs/reference/rest/v1/videos/annotate#feature)
feature and annotate these entities with labels (tags). This feature identifies
objects, locations, activities, animal species, products, and more.

Label detection differs from [Object tracking](https://docs.cloud.google.com/video-intelligence/docs/object-tracking).
Unlike object tracking, label detection provides labels for the entire frame
(without bounding boxes).

For example, for a video of a train at a crossing, the Video Intelligence API
returns labels such as "train", "transportation", "railroad crossing",
and so on. Each label includes a time segment with the time offset (timestamp)
for the entity's appearance from the beginning of the video.
Each annotation also contains additional information including an entity
id that you can use to find more information about the
entity in the [Google Knowledge Graph Search API](https://developers.google.com/knowledge-graph/).

Each entity returned can also include associated
category entities in the `categoryEntities` field. For example the
"Terrier" entity label has a category of "Dog". Category entities have a
hierarchy. For example, the "Dog" category is a child of the "Mammal"
category in the hierarchy. For a list of the common category entities that the
Video Intelligence uses, see
[entry-level-categories.json](https://docs.cloud.google.com/static/video-intelligence/docs/entry-level-categories.json).

The analysis can be compartmentalized as follows:

- Segment level:  
  User-selected segments of a video can be specified for analysis by stipulating beginning and ending timestamps for the purposes of annotation (see [VideoSegment](https://docs.cloud.google.com/video-intelligence/docs/reference/rest/v1/videos/annotate#videosegment)). Entities are then identified and labeled within each segment. If no segments are specified, the whole video is treated as one segment.

  <br />

  <br />

- Shot level:   
  Shots (also known as a *scene*) are automatically detected within every segment (or video). Entities are then identified and labeled within each scene. For details, see [Shot change detection](https://docs.cloud.google.com/video-intelligence/docs/feature-label-detection#shot-change)
- Frame level:   
  Entities are identified and labeled within each frame (with one frame per second sampling).

<br />

To detect labels in a video, call the
[`annotate`](https://docs.cloud.google.com/video-intelligence/docs/reference/rest/v1/videos/annotate)
method and specify
[`LABEL_DETECTION`](https://docs.cloud.google.com/video-intelligence/docs/reference/rest/v1/videos#Feature)
in the `features` field.

See
[Analyzing Videos for Labels](https://docs.cloud.google.com/video-intelligence/docs/analyze-labels) and
[Label Detection Tutorial](https://docs.cloud.google.com/video-intelligence/docs/label-tutorial).

# Video Intelligence API Visualizer

Check out the [Video Intelligence API visualizer](https://zackakil.github.io/video-intelligence-api-visualiser/#Label%20Detection) to see this feature in action.