Gemini Transcribe Live

Use Gemini to instantly convert streaming audio into text for live conversations. Gemini transcribe live offers reliable instant intermediate transcriptions. This feature offers instant and reliable intermediate transcriptions because it is optimized for real-time streaming transcription through the StreamingAnalyzeContent (SAC) and BidiStreamingAnalyzeContent (Bidi-SAC) APIs. To ensure optimal performance when using speech adaptation, use up to 100 phrase sets or adaptation terms.

Set up guide

Follow these steps to enable Gemini transcribe live.

Console

When you create or update a conversation profile in the Agent Assist console, follow these steps to use the Gemini model for Speech-to-Text API.

  1. Navigate to Conversation profiles and select your profile.
  2. In the Speech to Text Config section, choose the Gemini transcribe live model.
  3. Set the Language.
  4. Set the following parameters:
    1. Audio encoding: Linear 16
    2. Sample rate: 16000
  5. Select Use Long Form Model for AA Telephony SipRec Integration.
  6. (Optional) Configure Phrases for speech adaptation to improve accuracy with model adaptation. Each term must be less than or equal to 100 terms and each term must be less than or equal to 100 characters.

REST API

You can call the API directly to create or update a conversation profile, as shown in the following example.

Example configuration:

{
"name": "projects/PROJECT_ID/locations/global/conversationProfiles/CONVERSATION_PROFILE_ID",
    "display_name": "CONVERSATION_PROFILE_NAME",
    "stt_config": {
        "use_long_form_model": true,
        "audio_encoding": "AUDIO_ENCODING_LINEAR_16",
        "sample_rate_hertz": 16000,
        "use_gemini_asr": true
    },
    "language_code": "en-US"
}

Regional and language support

Gemini transcribe live is available in the global region only. See the supported languages list for available languages.