教學課程:使用控制台執行評估

瞭解如何透過 Google Cloud 控制台開始使用 Gen AI Evaluation Service。

事前準備

  1. 登入 Google Cloud 帳戶。如果您未曾使用過 Google Cloud,歡迎 建立帳戶,親自體驗實際使用我們產品的成效。新客戶還能獲得價值 $300 美元的免費抵免額,用於執行、測試及部署工作負載。
  2. In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

    Roles required to select or create a project

    • Select a project: Selecting a project doesn't require a specific IAM role—you can select any project that you've been granted a role on.
    • Create a project: To create a project, you need the Project Creator role (roles/resourcemanager.projectCreator), which contains the resourcemanager.projects.create permission. Learn how to grant roles.

    Go to project selector

  3. Verify that billing is enabled for your Google Cloud project.

  4. Make sure that you have the following role or roles on the project: Storage Admin

    Check for the roles

    1. In the Google Cloud console, go to the IAM page.

      Go to IAM
    2. Select the project.
    3. In the Principal column, find all rows that identify you or a group that you're included in. To learn which groups you're included in, contact your administrator.

    4. For all rows that specify or include you, check the Role column to see whether the list of roles includes the required roles.

    Grant the roles

    1. In the Google Cloud console, go to the IAM page.

      Go to IAM
    2. Select the project.
    3. Click Grant access.
    4. In the New principals field, enter your user identifier. This is typically the email address for a Google Account.

    5. Click Select a role, then search for the role.
    6. To grant additional roles, click Add another role and add each additional role.
    7. Click Save.
  5. In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

    Roles required to select or create a project

    • Select a project: Selecting a project doesn't require a specific IAM role—you can select any project that you've been granted a role on.
    • Create a project: To create a project, you need the Project Creator role (roles/resourcemanager.projectCreator), which contains the resourcemanager.projects.create permission. Learn how to grant roles.

    Go to project selector

  6. Verify that billing is enabled for your Google Cloud project.

  7. Make sure that you have the following role or roles on the project: Storage Admin

    Check for the roles

    1. In the Google Cloud console, go to the IAM page.

      Go to IAM
    2. Select the project.
    3. In the Principal column, find all rows that identify you or a group that you're included in. To learn which groups you're included in, contact your administrator.

    4. For all rows that specify or include you, check the Role column to see whether the list of roles includes the required roles.

    Grant the roles

    1. In the Google Cloud console, go to the IAM page.

      Go to IAM
    2. Select the project.
    3. Click Grant access.
    4. In the New principals field, enter your user identifier. This is typically the email address for a Google Account.

    5. Click Select a role, then search for the role.
    6. To grant additional roles, click Add another role and add each additional role.
    7. Click Save.

評估模型

如要評估模型,請按照下列步驟操作:

  1. 前往 Google Cloud 控制台的「Gen AI Evaluation」頁面。

    前往「Evaluation」(評估) 頁面

  2. 按一下「New evaluation」(新增評估),開啟評估頁面。

  3. 選取來源,載入評估用的資料集:

    • 如要上傳本機 CSV 或 JSONL 檔案,請選取「上傳檔案」。資料集必須包含提示詞或記錄,以便用於提示詞範本,以及模型回覆 (選用)。上限為 200 列。

    • 如要根據提示詞範本生成提示詞,請選取「Generate data」(生成資料)。Gen AI Evaluation Service 會生成並填入您在建立資料集時,於提示詞範本中定義的變數。如要進一步瞭解如何編寫提示詞範本,請參閱「使用提示詞範本」。

      1. 在「Prompt template」(提示詞範本) 欄位中,輸入提示詞範本和變數。

      2. 如要為每個變數新增說明,或指定要生成的樣本數量,請展開「Define variables and sample size」(定義變數和樣本大小)。

      3. 點按「生成資料集」,生成提示。

  4. 根據提示詞生成及評估回覆:

    1. 在「Evaluation candidates」(評估候選項目) 部分,按一下「Add evaluation candidate」(新增評估候選項目)。如果已有候選項目,請按一下 「Edit」(編輯),定義要評估的提示和回覆。舉例來說,您可以指定上傳檔案或生成資料中的提示或回覆。

    2. 如要比較多個候選項目,請按一下「新增候選比較項目」。

    3. 在「Metrics」(指標) 部分,至少新增一項指標,為候選回覆的品質評分。如要進一步瞭解指標類型,請參閱 Gen AI Evaluation Service 服務總覽頁面的「評估指標」一節。

    對於部分自動調整評分量表,您可以展開「Advanced」(進階) 並提供自訂指示 (例如 Evaluate the dataset on cultural sensitivity),引導系統根據每個提示生成評分量表。

    1. 在「Name and storage configuration」(名稱和儲存空間設定) 區段中,指定評估作業的名稱,以及用於儲存評估結果的 Cloud Storage bucket。
  5. 按一下「Evaluate」(評估)。

查看評估結果

如要查看評估結果,請按照下列步驟操作:

  1. 前往 Google Cloud 控制台的「GenAI Evaluation」(GenAI 評估) 頁面。

    前往「Evaluation」(評估) 頁面

  2. 按一下「Evaluation name」(評估作業名稱)。

    評估資料集中的每個提示都會顯示回覆和評估結果。

評估合作夥伴模型

您可以使用 Gen AI Evaluation Service 評估下列合作夥伴模型:

  • Anthropic
  • Llama

Gemini Enterprise Agent Platform Model Garden 支援合作夥伴模型。您必須先在 Model Garden 啟用合作夥伴模型,才能選取該模型進行評估。如要評估合作夥伴模型,請在設定評估作業時,從模型選取選單中選取該模型。

定價

評估第三方模型的費用,取決於在 Gemini Enterprise Agent Platform Model Garden 進行模型推論時產生的費用。請參閱 Agent Platform 生成式 AI 的定價頁面。

後續步驟