透過模型調整機制來改善語音轉錄結果

總覽

您可以使用模型調整功能，讓 Speech-to-Text 以比其他系統建議選項更高的頻率，辨識特定字詞或詞組。舉例來說，假設音訊資料經常包含「天氣」一詞，當 Speech-to-Text 遇到「weather」這個字詞時，您希望它將這個字詞轉錄為「weather」的頻率高於「whether」。在這種情況下，您可以使用模型調整功能，讓 Speech-to-Text 傾向辨識「weather」。

模型調整功能在下列用途中特別實用：

• 提升音訊資料中經常出現的字詞和詞組準確度。舉例來說，您可以提醒辨識模型使用者通常會說的語音指令。

  • 擴充 Speech-to-Text 辨識的字詞。語音轉文字功能包含非常龐大的詞彙。不過，如果音訊資料經常包含一般語言中不常使用的字詞 (例如專有名詞或特定領域的字詞)，您可以使用模型調整功能新增這些字詞。

  • 在提供的音訊含有雜訊或不夠清晰時，提高語音轉錄的準確度。

您也可以使用模型調整提升功能，微調辨識模型的偏誤。

提升字詞和詞組辨識準確度

如要提高 Speech-to-Text 在轉錄音訊資料時辨識出「weather」一詞的機率，您可以在 SpeechAdaptation 資源的 PhraseSet 物件中傳遞「weather」這個單字。

如果提供多字詞組，Speech-to-Text 較有可能依序辨識這些字詞。提供詞組也會提高辨識詞組部分內容 (包括個別字詞) 的機率。如要瞭解這些片語的數量和大小限制，請參閱內容限制頁面。

使用類別提升辨識準確度

類別代表自然語言中常見的概念，例如貨幣單位和日曆日期。如果有一大組字詞對應至常見概念，但並非一律包含相同字詞或詞組，您可以使用類別提升轉錄準確度。

舉例來說，假設音訊資料包含使用者說出街道地址的錄音檔，你可能錄到有人說「我家是 123 Main Street，左邊第四間房子」。在本例中，您希望 Speech-to-Text 將第一組數字序列「123」辨識為地址，而非序數「一百二十三」。不過，並非所有人都住在「中正路 123 號」。在 PhraseSet 資源中列出所有可能的街道地址並不切實際。您可以改用類別，指出無論實際數字為何，都應辨識為門牌號碼。在這個範例中，語音轉文字服務可以更準確地轉錄「123 Main Street」和「987 Grand Boulevard」等詞組，因為這兩者都會辨識為地址號碼。

類別權杖

如要在模型調整中使用類別，請在 PhraseSet 資源的 phrases 欄位中加入類別權杖。請參閱支援的類別符記清單，瞭解您的語言支援哪些符記。舉例來說，如要提升來源音訊中地址號碼的轉錄品質，請在 $ADDRESSNUM PhraseSet.

您可以在 phrases 陣列中將類別當做獨立項目使用，也可以在較長的多字詞組中嵌入一或多個類別符記。舉例來說，您可以在字串中加入類別權杖，在較長的片語中指出地址號碼：["my address is $ADDRESSNUM"]。不過，如果音訊包含類似但不完全相同的詞組 (例如「我在 123 Main Street」)，這類詞組就無法提供協助。為協助辨識類似詞組，請務必另外加入類別符記本身：["my address is $ADDRESSNUM", "$ADDRESSNUM"]。如果使用無效或格式錯誤的類別符記，Speech-to-Text 會忽略該符記，不會觸發錯誤，但仍會使用其餘片語做為情境。

注意： 課程適用情形因轉錄模型和語言而異。

自訂類別

您也可以建立自己的CustomClass，也就是由自訂清單組成的類別，清單內含相關的項目或值。舉例來說，您想轉錄的音訊資料可能包含數百家區域餐廳的名稱。餐廳名稱在一般語音中相對罕見，因此辨識模型較不可能將其選為「正確」答案。您可以自訂調整辨識模型，這些名稱出現在音訊中時，就能偏向正確的辨識結果。

如要使用自訂類別，請建立 CustomClass 資源，其中包含每個餐廳名稱做為 ClassItem。自訂類別的運作方式與預先建構的類別權杖相同。phrase 可以同時包含預先建構的類別權杖和自訂類別。

使用提升功能微調轉錄結果

預設情況下，模型調整項的影響相對較小，尤其是單字詞組。模型調整功能的提升功能可讓您為某些詞組指派較高的權重，藉此提高辨識模型偏誤。如果符合下列所有情況，建議您採用加速功能：

1. 您已導入模型調整功能。
2. 您想進一步調整模型調整機制對轉錄結果的影響程度。如要查看您的語言是否支援強化功能，請參閱語言支援頁面。

舉例來說，許多錄音檔中都有人詢問「進入郡集市的票價」，且「集市」一詞的出現頻率高於「票價」。在這種情況下，您可以透過模型調整，在 PhraseSet 資源中將「fair」和「fare」新增為 phrases，提高模型辨識這兩個字的可能性。這會指示 Speech-to-Text 優先辨識「fair」和「fare」，而非「hare」或「lair」。

不過，由於「fair」在音訊中出現的頻率較高，因此系統應更常辨識出「fair」而非「fare」。您可能已使用 Speech-to-Text API 轉錄音訊，但發現系統辨識正確字詞 (「fair」) 時發生大量錯誤。在這種情況下，您可以使用提升功能，為「fair」指派比「fare」更高的提升值。「fair」的加權值較高，因此 Speech-to-Text API 會偏向選擇「fair」，而非「fare」。如果沒有提升值，辨識模型會以相同機率辨識「fair」和「fare」。

基本提升

使用提升功能時，您會在 PhraseSet 資源中為 phrase 項目指派加權值。Speech-to-Text 會參考這個加權值，為音訊資料中的字詞選取可能的轉錄內容。增強值越高，Speech-to-Text 從可能選項中選擇該字詞或詞組的機率便越高。

如果為多字詞組指派增強值，系統會對整個詞組套用增強效果，且只會對整個詞組套用。舉例來說，您想為「我最喜歡的美國自然史博物館展覽是藍鯨」這個片語指派提升值。如果將該詞組新增至 phrase 物件並指派提升值，辨識模型就更有可能逐字辨識出整個詞組。

如果提升多字詞組的權重後，仍未獲得所需結果，建議您將組成該詞組的所有雙連字 (2 個字，依序排列) 新增為 phrase 項目，並為每個項目指派權重值。延續先前的例子，您可以考慮加入其他雙連詞和尾連詞 (超過兩個字)，例如「我最喜歡」、「我最喜歡的展覽」、「最喜歡的展覽」、「我最喜歡的美國自然史博物館展覽」、「美國自然史博物館」和「藍鯨」。這樣一來，STT 辨識模型就更有可能辨識出音訊中的相關詞組，這些詞組包含原始加權詞組的部分內容，但並非完全相同。

設定提升值

增幅值必須是大於 0 的浮點值。實際的升幅值上限為 20。為獲得最佳結果，請上下調整提升值，直到取得準確的轉錄結果。

提高提升值可減少偽陰性，也就是字詞或詞組出現在音訊中，但 Speech-to-Text 未正確辨識的情況。不過，提高信心分數也可能增加誤判的機率，也就是說，即使音訊中沒有出現該字詞或詞組，轉錄稿中仍可能出現。

使用模型調整功能的用途範例

以下範例逐步說明如何使用模型調整功能，轉錄某人說「The word is fare」的錄音內容。在本例中，如果沒有語音適應功能，語音轉文字服務會辨識出「fair」一字。使用語音調整功能後，Speech-to-Text 就能辨識出「fare」這個字。

事前準備

登入 Google Cloud 帳戶。如果您是 Google Cloud新手，歡迎 建立帳戶，親自體驗產品的實際應用成效。新客戶還能獲得價值 $300 美元的免費抵免額，能用於執行、測試及部署工作負載。

In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

Roles required to select or create a project

• Select a project: Selecting a project doesn't require a specific IAM role—you can select any project that you've been granted a role on.
• Create a project: To create a project, you need the Project Creator role (roles/resourcemanager.projectCreator), which contains the resourcemanager.projects.create permission. Learn how to grant roles.

Note: If you don't plan to keep the resources that you create in this procedure, create a project instead of selecting an existing project. After you finish these steps, you can delete the project, removing all resources associated with the project.

Go to project selector

Verify that billing is enabled for your Google Cloud project.

Enable the Speech-to-Text APIs, if any are not already enabled.

Roles required to enable APIs

To enable APIs, you need the serviceusage.services.enable permission. If you created the project, then you likely already have this permission through the Owner role (roles/owner). Otherwise, you can get this permission through the Service Usage Admin role (roles/serviceusage.serviceUsageAdmin). Learn how to grant roles.

Enable the APIs

Make sure that you have the following role or roles on the project: Cloud Speech Administrator

Check for the roles

1. In the Google Cloud console, go to the IAM page.

   Go to IAM
2. Select the project.

3. In the Principal column, find all rows that identify you or a group that you're included in. To learn which groups you're included in, contact your administrator.

4. For all rows that specify or include you, check the Role column to see whether the list of roles includes the required roles.

Grant the roles

1. In the Google Cloud console, go to the IAM page.

   Go to IAM
2. Select the project.
3. Click Grant access.

4. In the New principals field, enter your user identifier. This is typically the email address for a Google Account.

5. Click Select a role, then search for the role.
6. To grant additional roles, click Add another role and add each additional role.
7. Click Save.

Install the Google Cloud CLI.

Note: If you installed the gcloud CLI previously, make sure you have the latest version by running gcloud components update.

If you're using an external identity provider (IdP), you must first sign in to the gcloud CLI with your federated identity.

To initialize the gcloud CLI, run the following command:

gcloud init

In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

Roles required to select or create a project

• Select a project: Selecting a project doesn't require a specific IAM role—you can select any project that you've been granted a role on.
• Create a project: To create a project, you need the Project Creator role (roles/resourcemanager.projectCreator), which contains the resourcemanager.projects.create permission. Learn how to grant roles.

Note: If you don't plan to keep the resources that you create in this procedure, create a project instead of selecting an existing project. After you finish these steps, you can delete the project, removing all resources associated with the project.

Go to project selector

Verify that billing is enabled for your Google Cloud project.

Enable the Speech-to-Text APIs, if any are not already enabled.

Roles required to enable APIs

To enable APIs, you need the serviceusage.services.enable permission. If you created the project, then you likely already have this permission through the Owner role (roles/owner). Otherwise, you can get this permission through the Service Usage Admin role (roles/serviceusage.serviceUsageAdmin). Learn how to grant roles.

Enable the APIs

Make sure that you have the following role or roles on the project: Cloud Speech Administrator

Check for the roles

1. In the Google Cloud console, go to the IAM page.

   Go to IAM
2. Select the project.

3. In the Principal column, find all rows that identify you or a group that you're included in. To learn which groups you're included in, contact your administrator.

4. For all rows that specify or include you, check the Role column to see whether the list of roles includes the required roles.

Grant the roles

1. In the Google Cloud console, go to the IAM page.

   Go to IAM
2. Select the project.
3. Click Grant access.

4. In the New principals field, enter your user identifier. This is typically the email address for a Google Account.

5. Click Select a role, then search for the role.
6. To grant additional roles, click Add another role and add each additional role.
7. Click Save.

Install the Google Cloud CLI.

Note: If you installed the gcloud CLI previously, make sure you have the latest version by running gcloud components update.

If you're using an external identity provider (IdP), you must first sign in to the gcloud CLI with your federated identity.

To initialize the gcloud CLI, run the following command:

gcloud init

用戶端程式庫可以使用應用程式預設憑證，輕鬆向 Google API 進行驗證，並向這些 API 傳送要求。有了應用程式預設憑證，您可以在本機測試及部署應用程式，不必變更基礎程式碼。詳情請參閱「 進行驗證以使用用戶端程式庫」一文。

1. 如果您使用的是本機殼層，請為使用者帳戶建立本機驗證憑證：

   gcloud auth application-default login

   如果您使用 Cloud Shell，則不需要執行這項操作。

   如果系統傳回驗證錯誤，且您使用的是外部識別資訊提供者 (IdP)，請確認您已 使用聯合身分登入 gcloud CLI。

此外，請務必安裝用戶端程式庫。

使用 PhraseSet 提升轉錄品質

1. 以下範例會使用「fare」這個詞組建構 PhraseSet，並在辨識要求中將其新增為 inline_phrase_set：

Python

import os

from google.cloud.speech_v2 import SpeechClient
from google.cloud.speech_v2.types import cloud_speech

PROJECT_ID = os.getenv("GOOGLE_CLOUD_PROJECT")


def adaptation_v2_inline_phrase_set(audio_file: str) -> cloud_speech.RecognizeResponse:
    """Enhances speech recognition accuracy using an inline phrase set.
    The inline custom phrase set helps the recognizer produce more accurate transcriptions for specific terms.
    Phrases are given a boost to increase their chances of being recognized correctly.
    Args:
        audio_file (str): Path to the local audio file to be transcribed.
    Returns:
        cloud_speech.RecognizeResponse: The full response object which includes the transcription results.
    """

    # Instantiates a client
    client = SpeechClient()

    # Reads a file as bytes
    with open(audio_file, "rb") as f:
        audio_content = f.read()

    # Build inline phrase set to produce a more accurate transcript
    phrase_set = cloud_speech.PhraseSet(
        phrases=[{"value": "fare", "boost": 10}, {"value": "word", "boost": 20}]
    )
    adaptation = cloud_speech.SpeechAdaptation(
        phrase_sets=[
            cloud_speech.SpeechAdaptation.AdaptationPhraseSet(
                inline_phrase_set=phrase_set
            )
        ]
    )
    config = cloud_speech.RecognitionConfig(
        auto_decoding_config=cloud_speech.AutoDetectDecodingConfig(),
        adaptation=adaptation,
        language_codes=["en-US"],
        model="short",
    )

    # Prepare the request which includes specifying the recognizer, configuration, and the audio content
    request = cloud_speech.RecognizeRequest(
        recognizer=f"projects/{PROJECT_ID}/locations/global/recognizers/_",
        config=config,
        content=audio_content,
    )

    # Transcribes the audio into text
    response = client.recognize(request=request)

    for result in response.results:
        print(f"Transcript: {result.alternatives[0].transcript}")

    return response

1. 這個範例會建立具有相同詞組的 PhraseSet 資源，然後在辨識要求中參照該資源：

Python

import os

from google.cloud.speech_v2 import SpeechClient
from google.cloud.speech_v2.types import cloud_speech

PROJECT_ID = os.getenv("GOOGLE_CLOUD_PROJECT")


def adaptation_v2_phrase_set_reference(
    audio_file: str,
    phrase_set_id: str,
) -> cloud_speech.RecognizeResponse:
    """Transcribe audio files using a PhraseSet.
    Args:
        audio_file (str): Path to the local audio file to be transcribed.
        phrase_set_id (str): The unique ID of the PhraseSet to use.
    Returns:
        cloud_speech.RecognizeResponse: The full response object which includes the transcription results.
    """

    # Instantiates a client
    client = SpeechClient()

    # Reads a file as bytes
    with open(audio_file, "rb") as f:
        audio_content = f.read()

    # Creating operation of creating the PhraseSet on the cloud.
    operation = client.create_phrase_set(
        parent=f"projects/{PROJECT_ID}/locations/global",
        phrase_set_id=phrase_set_id,
        phrase_set=cloud_speech.PhraseSet(phrases=[{"value": "fare", "boost": 10}]),
    )
    phrase_set = operation.result()

    # Add a reference of the PhraseSet into the recognition request
    adaptation = cloud_speech.SpeechAdaptation(
        phrase_sets=[
            cloud_speech.SpeechAdaptation.AdaptationPhraseSet(
                phrase_set=phrase_set.name
            )
        ]
    )

    # Automatically detect audio encoding. Use "short" model for short utterances.
    config = cloud_speech.RecognitionConfig(
        auto_decoding_config=cloud_speech.AutoDetectDecodingConfig(),
        adaptation=adaptation,
        language_codes=["en-US"],
        model="short",
    )
    #  Prepare the request which includes specifying the recognizer, configuration, and the audio content
    request = cloud_speech.RecognizeRequest(
        recognizer=f"projects/{PROJECT_ID}/locations/global/recognizers/_",
        config=config,
        content=audio_content,
    )
    # Transcribes the audio into text
    response = client.recognize(request=request)

    for result in response.results:
        print(f"Transcript: {result.alternatives[0].transcript}")

    return response

使用 CustomClass 提升轉錄結果品質

1. 以下範例會建構 CustomClass，其中包含「fare」項目和「fare」名稱。然後在辨識要求中，於 inline_phrase_set 內參照 CustomClass：

Python

import os

from google.cloud.speech_v2 import SpeechClient
from google.cloud.speech_v2.types import cloud_speech

PROJECT_ID = os.getenv("GOOGLE_CLOUD_PROJECT")


def adaptation_v2_inline_custom_class(
    audio_file: str,
) -> cloud_speech.RecognizeResponse:
    """Transcribe audio file using inline custom class.
    The inline custom class helps the recognizer produce more accurate transcriptions for specific terms.
    Args:
        audio_file (str): Path to the local audio file to be transcribed.
    Returns:
        cloud_speech.RecognizeResponse: The response object which includes the transcription results.
    """
    # Instantiates a client
    client = SpeechClient()

    # Reads a file as bytes
    with open(audio_file, "rb") as f:
        audio_content = f.read()

    # Define an inline custom class to enhance recognition accuracy with specific items like "fare" etc.
    custom_class_name = "your-class-name"
    custom_class = cloud_speech.CustomClass(
        name=custom_class_name,
        items=[{"value": "fare"}],
    )

    # Build inline phrase set to produce a more accurate transcript
    phrase_set = cloud_speech.PhraseSet(
        phrases=[{"value": custom_class_name, "boost": 20}]
    )
    adaptation = cloud_speech.SpeechAdaptation(
        phrase_sets=[
            cloud_speech.SpeechAdaptation.AdaptationPhraseSet(
                inline_phrase_set=phrase_set
            )
        ],
        custom_classes=[custom_class],
    )
    config = cloud_speech.RecognitionConfig(
        auto_decoding_config=cloud_speech.AutoDetectDecodingConfig(),
        adaptation=adaptation,
        language_codes=["en-US"],
        model="short",
    )

    # Prepare the request which includes specifying the recognizer, configuration, and the audio content
    request = cloud_speech.RecognizeRequest(
        recognizer=f"projects/{PROJECT_ID}/locations/global/recognizers/_",
        config=config,
        content=audio_content,
    )

    # Transcribes the audio into text
    response = client.recognize(request=request)

    for result in response.results:
        print(f"Transcript: {result.alternatives[0].transcript}")

    return response

1. 這個範例會建立具有相同項目的 CustomClass 資源。然後建立 PhraseSet 資源，其中包含參照 CustomClass 資源名稱的片語。然後在辨識要求中參照 PhraseSet 資源：

Python

import os

from google.cloud.speech_v2 import SpeechClient
from google.cloud.speech_v2.types import cloud_speech

PROJECT_ID = os.getenv("GOOGLE_CLOUD_PROJECT")


def adaptation_v2_custom_class_reference(
    audio_file: str, phrase_set_id: str, custom_class_id: str
) -> cloud_speech.RecognizeResponse:
    """Transcribe audio file using a custom class.
    Args:
        audio_file (str): Path to the local audio file to be transcribed.
        phrase_set_id (str): The unique ID of the phrase set to use.
        custom_class_id (str): The unique ID of the custom class to use.
    Returns:
        cloud_speech.RecognizeResponse: The full response object which includes the transcription results.
    """
    # Instantiates a speech client
    client = SpeechClient()

    # Reads a file as bytes
    with open(audio_file, "rb") as f:
        audio_content = f.read()

    # Create a custom class to improve recognition accuracy for specific terms
    custom_class = cloud_speech.CustomClass(items=[{"value": "fare"}])
    operation = client.create_custom_class(
        parent=f"projects/{PROJECT_ID}/locations/global",
        custom_class_id=custom_class_id,
        custom_class=custom_class,
    )
    custom_class = operation.result()

    # Create a persistent PhraseSet to reference in a recognition request
    created_phrase_set = cloud_speech.PhraseSet(
        phrases=[
            {
                "value": f"${{{custom_class.name}}}",
                "boost": 20,
            },  # Using custom class reference
        ]
    )
    operation = client.create_phrase_set(
        parent=f"projects/{PROJECT_ID}/locations/global",
        phrase_set_id=phrase_set_id,
        phrase_set=created_phrase_set,
    )
    phrase_set = operation.result()

    # Add a reference of the PhraseSet into the recognition request
    adaptation = cloud_speech.SpeechAdaptation(
        phrase_sets=[
            cloud_speech.SpeechAdaptation.AdaptationPhraseSet(
                phrase_set=phrase_set.name
            )
        ]
    )
    # Automatically detect the audio's encoding with short audio model
    config = cloud_speech.RecognitionConfig(
        auto_decoding_config=cloud_speech.AutoDetectDecodingConfig(),
        adaptation=adaptation,
        language_codes=["en-US"],
        model="short",
    )

    # Create a custom class to reference in a PhraseSet
    request = cloud_speech.RecognizeRequest(
        recognizer=f"projects/{PROJECT_ID}/locations/global/recognizers/_",
        config=config,
        content=audio_content,
    )

    # Transcribes the audio into text
    response = client.recognize(request=request)

    for result in response.results:
        print(f"Transcript: {result.alternatives[0].transcript}")

    return response

清除所用資源

如要避免系統向您的 Google Cloud 帳戶收取本頁所用資源的費用，請按照下列步驟操作。

1. 選用：撤銷您建立的驗證憑證，並刪除本機憑證檔案。

   gcloud auth application-default revoke

2. 選用：從 gcloud CLI 撤銷憑證。

   gcloud auth revoke

控制台

注意：刪除專案會有以下後果：

• 專案中的所有內容都會遭到刪除。如果使用現有專案來進行本文中的任務，刪除專案將一併移除當中已完成的其他任務'。
• 自訂專案 ID 會消失。當您之前建立這個專案時，可能建立了想要在日後使用的自訂專案 ID。如要保留使用該專案 ID 的網址 (例如 appspot.com 網址)，請刪除在該專案中選取的資源，而不是刪除整個專案。

如果打算探索多種架構、教學課程或快速入門導覽課程，重複使用專案可避免超出專案配額限制。

前往 Google Cloud 控制台的「Manage resources」(管理資源) 頁面。

前往「Manage resources」(管理資源)

在專案清單中選取要刪除的專案，然後點選「Delete」(刪除)。

在對話方塊中輸入專案 ID，然後按一下 [Shut down] (關閉) 以刪除專案。

gcloud

注意：刪除專案會有以下後果：

• 專案中的所有內容都會遭到刪除。如果使用現有專案來進行本文中的任務，刪除專案將一併移除當中已完成的其他任務'。
• 自訂專案 ID 會消失。當您之前建立這個專案時，可能建立了想要在日後使用的自訂專案 ID。如要保留使用該專案 ID 的網址 (例如 appspot.com 網址)，請刪除在該專案中選取的資源，而不是刪除整個專案。

如果打算探索多種架構、教學課程或快速入門導覽課程，重複使用專案可避免超出專案配額限制。

刪除 Google Cloud 專案：

gcloud projects delete PROJECT_ID

後續步驟

• 查看支援的類別符記清單。
• 瞭解如何轉錄短音訊檔案。
• 瞭解如何轉錄串流音訊。
• 瞭解如何轉錄長音訊檔案。