使用 Agent Platform SDK 中的生成式 AI 客户端评估生成式 AI 代理

构建并评估生成式 AI 模型后，您便可以使用该模型构建代理，例如聊天机器人。借助 Gen AI 评估，您可以衡量该代理在您的应用场景中完成任务和实现目标的能力。

本页介绍了如何创建和部署基本代理，以及如何使用 Gen AI 评估来评估代理：

准备工作

  1. 登录您的 Google Cloud 账号。如果您是 Google Cloud的新用户， 请创建账号，以评估我们的产品在 真实场景中的表现。新客户还可以获得 300 美元的免费抵用金，用于 运行、测试和部署工作负载。

    In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

    Roles required to select or create a project

    • Select a project: Selecting a project doesn't require a specific IAM role—you can select any project that you've been granted a role on.
    • Create a project: To create a project, you need the Project Creator role (roles/resourcemanager.projectCreator), which contains the resourcemanager.projects.create permission. Learn how to grant roles.

    Go to project selector

    Verify that billing is enabled for your Google Cloud project.

    In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

    Roles required to select or create a project

    • Select a project: Selecting a project doesn't require a specific IAM role—you can select any project that you've been granted a role on.
    • Create a project: To create a project, you need the Project Creator role (roles/resourcemanager.projectCreator), which contains the resourcemanager.projects.create permission. Learn how to grant roles.

    Go to project selector

    Verify that billing is enabled for your Google Cloud project.

  2. 安装 Agent Platform SDK for Python：

    %pip install google-cloud-aiplatform[adk,agent_engines]
%pip install --upgrade --force-reinstall -q google-cloud-aiplatform[evaluation]

  3. 设置凭据。如果您是在 Colaboratory 中运行本教程，请运行以下命令：

    from google.colab import auth
auth.authenticate_user()

    对于其他环境，请参阅向 Agent Platform进行身份验证。

  4. 在 Agent Platform SDK 中初始化 GenAI 客户端：

    import vertexai
from vertexai import Client
from google.genai import types as genai_types

GCS_DEST = "gs://BUCKET_NAME/output-path"
vertexai.init(
    project=PROJECT_ID,
    location=LOCATION,
)

client = Client(
    project=PROJECT_ID,
    location=LOCATION,
    http_options=genai_types.HttpOptions(api_version="v1beta1"),
  )

    替换以下内容：

    • BUCKET_NAME：Cloud Storage 存储桶名称。如需详细了解如何创建 存储桶，请参阅创建 存储桶。

    • PROJECT_ID：您的项目 ID。

    • LOCATION：您选择的区域。

开发代理

通过定义模型、指令和工具集来开发智能体开发套件 (ADK) 代理。如需详细了解如何开发代理，请参阅开发智能体开发套件代理。

from google.adk import Agent

# Define Agent Tools
def search_products(query: str):
    """Searches for products based on a query."""
    # Mock response for demonstration
    if "headphones" in query.lower():
        return {"products": [{"name": "Wireless Headphones", "id": "B08H8H8H8H"}]}
    else:
        return {"products": []}

def get_product_details(product_id: str):
    """Gets the details for a given product ID."""
    if product_id == "B08H8H8H8H":
        return {"details": "Noise-cancelling, 20-hour battery life."}
    else:
        return {"error": "Product not found."}

def add_to_cart(product_id: str, quantity: int):
    """Adds a specified quantity of a product to the cart."""
    return {"status": f"Added {quantity} of {product_id} to cart."}

# Define Agent
my_agent = Agent(
    model="gemini-2.5-flash",
    name='ecommerce_agent',
    instruction='You are an ecommerce expert',
    tools=[search_products, get_product_details, add_to_cart],
)

部署代理

将代理部署到 Agent Platform 运行时。完成此流程最多可能需要 10 分钟时间。从已部署的代理中检索资源名称。

def deploy_adk_agent(root_agent):
  """Deploy agent to agent engine.
  Args:
    root_agent: The ADK agent to deploy.
  """
  app = vertexai.agent_engines.AdkApp(
      agent=root_agent,
  )
  remote_app = client.agent_engines.create(
      agent=app,
      config = {
          "staging_bucket": gs://BUCKET_NAME,
          "requirements": ['google-cloud-aiplatform[adk,agent_engines]'],
          "env_vars": {"GOOGLE_CLOUD_AGENT_ENGINE_ENABLE_TELEMETRY": "true"}
      }
  )
  return remote_app

agent_engine = deploy_adk_agent(my_agent)
agent_engine_resource_name = agent_engine.api_resource.name

如需获取已部署到 Agent Platform 的代理列表， 请参阅管理已部署的代理。

生成回答

  1. 使用 run_inference() 为数据集生成模型回答：

    以 Pandas DataFrame 的形式准备数据集：提示应特定于您的代理。跟踪记录需要会话输入。如需了解详情，请参阅会话：跟踪个人 对话。

    import pandas as pd
from vertexai import types

session_inputs = types.evals.SessionInput(
    user_id="user_123",
    state={},
)
agent_prompts = [
    "Search for 'noise-cancelling headphones'.",
    "Show me the details for product 'B08H8H8H8H'.",
    "Add one pair of 'B08H8H8H8H' to my shopping cart.",
    "Find 'wireless earbuds' and then add the first result to my cart.",
    "I need a new laptop for work, can you find one with at least 16GB of RAM?",
]
agent_dataset = pd.DataFrame({
    "prompt": agent_prompts,
    "session_inputs": [session_inputs] * len(agent_prompts),
})

  2. 使用 run_inference() 生成模型回答：

    agent_dataset_with_inference = client.evals.run_inference(
    agent=agent_engine_resource_name,
    src=agent_dataset,
)

  3. 通过对 EvaluationDataset 对象调用 .show() 来直观呈现推理 结果 ，以便检查模型 输出以及原始提示和引用内容：

    agent_dataset_with_inference.show()

运行代理评估

运行 create_evaluation_run() 以评估代理回答。

  1. 使用内置的辅助函数检索 agent_info：

    agent_info = types.evals.AgentInfo.load_from_agent(
    my_agent,
    agent_engine_resource_name
)

  2. 使用特定于代理的基于自适应评分准则的 指标（FINAL_RESPONSE_QUALITY、TOOL_USE_QUALITY 和 HALLUCINATION）评估模型回答：

    evaluation_run = client.evals.create_evaluation_run(
    dataset=agent_dataset_with_inference,
    agent_info=agent_info,
    metrics=[
        types.RubricMetric.FINAL_RESPONSE_QUALITY,
        types.RubricMetric.TOOL_USE_QUALITY,
        types.RubricMetric.HALLUCINATION,
        types.RubricMetric.SAFETY,
    ],
    dest=GCS_DEST,
)

查看代理评估结果

您可以使用 Agent Platform SDK 查看评估结果。

检索评估运行，并通过调用 .show() 来显示 摘要指标和详细结果：

evaluation_run = client.evals.get_evaluation_run(
    name=evaluation_run.name,
    include_evaluation_items=True
)

evaluation_run.show()

详细结果还包括显示代理互动的跟踪记录。如需详细了解跟踪记录，请参阅跟踪代理。

后续步骤

试用以下智能体评估笔记本：