瞭解如何透過 Google Cloud 控制台開始使用 Gen AI Evaluation Service。
事前準備
- 登入 Google Cloud 帳戶。如果您未曾使用過 Google Cloud,歡迎 建立帳戶,親自體驗實際使用我們產品的成效。新客戶還能獲得價值 $300 美元的免費抵免額,用於執行、測試及部署工作負載。
-
In the Google Cloud console, on the project selector page, select or create a Google Cloud project.
Roles required to select or create a project
- Select a project: Selecting a project doesn't require a specific IAM role—you can select any project that you've been granted a role on.
-
Create a project: To create a project, you need the Project Creator role
(
roles/resourcemanager.projectCreator), which contains theresourcemanager.projects.createpermission. Learn how to grant roles.
-
Verify that billing is enabled for your Google Cloud project.
-
Make sure that you have the following role or roles on the project: Storage Admin
Check for the roles
-
In the Google Cloud console, go to the IAM page.
Go to IAM - Select the project.
-
In the Principal column, find all rows that identify you or a group that you're included in. To learn which groups you're included in, contact your administrator.
- For all rows that specify or include you, check the Role column to see whether the list of roles includes the required roles.
Grant the roles
-
In the Google Cloud console, go to the IAM page.
Go to IAM - Select the project.
- Click Grant access.
-
In the New principals field, enter your user identifier. This is typically the email address for a Google Account.
- Click Select a role, then search for the role.
- To grant additional roles, click Add another role and add each additional role.
- Click Save.
-
-
In the Google Cloud console, on the project selector page, select or create a Google Cloud project.
Roles required to select or create a project
- Select a project: Selecting a project doesn't require a specific IAM role—you can select any project that you've been granted a role on.
-
Create a project: To create a project, you need the Project Creator role
(
roles/resourcemanager.projectCreator), which contains theresourcemanager.projects.createpermission. Learn how to grant roles.
-
Verify that billing is enabled for your Google Cloud project.
-
Make sure that you have the following role or roles on the project: Storage Admin
Check for the roles
-
In the Google Cloud console, go to the IAM page.
Go to IAM - Select the project.
-
In the Principal column, find all rows that identify you or a group that you're included in. To learn which groups you're included in, contact your administrator.
- For all rows that specify or include you, check the Role column to see whether the list of roles includes the required roles.
Grant the roles
-
In the Google Cloud console, go to the IAM page.
Go to IAM - Select the project.
- Click Grant access.
-
In the New principals field, enter your user identifier. This is typically the email address for a Google Account.
- Click Select a role, then search for the role.
- To grant additional roles, click Add another role and add each additional role.
- Click Save.
-
評估模型
如要評估模型,請按照下列步驟操作:
前往 Google Cloud 控制台的「Gen AI Evaluation」頁面。
按一下「New evaluation」(新增評估),開啟評估頁面。
選取來源,載入評估用的資料集:
如要上傳本機 CSV 或 JSONL 檔案,請選取「上傳檔案」。資料集必須包含提示詞或記錄,以便用於提示詞範本,以及模型回覆 (選用)。上限為 200 列。
如要根據提示詞範本生成提示詞,請選取「Generate data」(生成資料)。Gen AI Evaluation Service 會生成並填入您在建立資料集時,於提示詞範本中定義的變數。如要進一步瞭解如何編寫提示詞範本,請參閱「使用提示詞範本」。
在「Prompt template」(提示詞範本) 欄位中,輸入提示詞範本和變數。
如要為每個變數新增說明,或指定要生成的樣本數量,請展開「Define variables and sample size」(定義變數和樣本大小)。
點按「生成資料集」,生成提示。
根據提示詞生成及評估回覆:
在「Evaluation candidates」(評估候選項目) 部分,按一下「Add evaluation candidate」(新增評估候選項目)。如果已有候選項目,請按一下 「Edit」(編輯),定義要評估的提示和回覆。舉例來說,您可以指定上傳檔案或生成資料中的提示或回覆。
如要比較多個候選項目,請按一下「新增候選比較項目」。
在「Metrics」(指標) 部分,至少新增一項指標,為候選回覆的品質評分。如要進一步瞭解指標類型,請參閱 Gen AI Evaluation Service 服務總覽頁面的「評估指標」一節。
對於部分自動調整評分量表,您可以展開「Advanced」(進階) 並提供自訂指示 (例如
Evaluate the dataset on cultural sensitivity),引導系統根據每個提示生成評分量表。- 在「Name and storage configuration」(名稱和儲存空間設定) 區段中,指定評估作業的名稱,以及用於儲存評估結果的 Cloud Storage bucket。
按一下「Evaluate」(評估)。
查看評估結果
如要查看評估結果,請按照下列步驟操作:
前往 Google Cloud 控制台的「GenAI Evaluation」(GenAI 評估) 頁面。
按一下「Evaluation name」(評估作業名稱)。
評估資料集中的每個提示都會顯示回覆和評估結果。
評估合作夥伴模型
您可以使用 Gen AI Evaluation Service 評估下列合作夥伴模型:
- Anthropic
- Llama
Gemini Enterprise Agent Platform Model Garden 支援合作夥伴模型。您必須先在 Model Garden 啟用合作夥伴模型,才能選取該模型進行評估。如要評估合作夥伴模型,請在設定評估作業時,從模型選取選單中選取該模型。
定價
評估第三方模型的費用,取決於在 Gemini Enterprise Agent Platform Model Garden 進行模型推論時產生的費用。請參閱 Agent Platform 生成式 AI 的定價頁面。