使用 Agent Platform SDK 中的生成式 AI 客户端评估生成式 AI 代理

构建并评估生成式 AI 模型后，您便可以使用该模型构建代理，例如聊天机器人。借助 Gemini Enterprise Agent Platform 评估，您可以衡量该智能体在您的应用场景中完成任务和实现目标的能力。

本页面介绍了如何创建和部署基本智能体，以及如何使用生成式 AI 评估来评估该智能体：

准备工作

  1. 登录您的 Google Cloud 账号。如果您是 Google Cloud新手，请 创建一个账号来评估我们的产品在实际场景中的表现。新客户还可获享 $300 赠金，用于运行、测试和部署工作负载。

    In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

    Roles required to select or create a project

    • Select a project: Selecting a project doesn't require a specific IAM role—you can select any project that you've been granted a role on.
    • Create a project: To create a project, you need the Project Creator role (roles/resourcemanager.projectCreator), which contains the resourcemanager.projects.create permission. Learn how to grant roles.

    Go to project selector

    Verify that billing is enabled for your Google Cloud project.

    In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

    Roles required to select or create a project

    • Select a project: Selecting a project doesn't require a specific IAM role—you can select any project that you've been granted a role on.
    • Create a project: To create a project, you need the Project Creator role (roles/resourcemanager.projectCreator), which contains the resourcemanager.projects.create permission. Learn how to grant roles.

    Go to project selector

    Verify that billing is enabled for your Google Cloud project.

  2. 安装 Agent Platform SDK for Python：

    %pip install google-cloud-aiplatform[adk,agent_engines]
%pip install --upgrade --force-reinstall -q google-cloud-aiplatform[evaluation]

  3. 设置凭据。如果您是在 Colaboratory 中运行本教程，请运行以下命令：

    from google.colab import auth
auth.authenticate_user()

    对于其他环境，请参阅向 Agent Platform 进行身份验证。

  4. 在 Agent Platform SDK 中初始化生成式 AI 客户端：

    import vertexai
from vertexai import Client
from google.genai import types as genai_types

GCS_DEST = "gs://BUCKET_NAME/output-path"
vertexai.init(
    project=PROJECT_ID,
    location=LOCATION,
)

client = Client(
    project=PROJECT_ID,
    location=LOCATION,
    http_options=genai_types.HttpOptions(api_version="v1beta1"),
  )

    替换以下内容：

    • BUCKET_NAME：Cloud Storage 存储桶名称。如需详细了解如何创建存储桶，请参阅创建存储桶。

    • PROJECT_ID：您的项目 ID。

    • LOCATION：您选择的区域。

开发代理

通过定义模型、指令和工具集来开发智能体开发套件 (ADK) 代理。如需详细了解如何开发代理，请参阅开发 ADK 代理。

from google.adk import Agent

# Define Agent Tools
def search_products(query: str):
    """Searches for products based on a query."""
    # Mock response for demonstration
    if "headphones" in query.lower():
        return {"products": [{"name": "Wireless Headphones", "id": "B08H8H8H8H"}]}
    else:
        return {"products": []}

def get_product_details(product_id: str):
    """Gets the details for a given product ID."""
    if product_id == "B08H8H8H8H":
        return {"details": "Noise-cancelling, 20-hour battery life."}
    else:
        return {"error": "Product not found."}

def add_to_cart(product_id: str, quantity: int):
    """Adds a specified quantity of a product to the cart."""
    return {"status": f"Added {quantity} of {product_id} to cart."}

# Define Agent
my_agent = Agent(
    model="gemini-2.5-flash",
    name='ecommerce_agent',
    instruction='You are an ecommerce expert',
    tools=[search_products, get_product_details, add_to_cart],
)

部署代理

将智能体部署到 Agent Platform Runtime。完成此流程最多可能需要 10 分钟时间。 从已部署的智能体中检索资源名称。

def deploy_adk_agent(root_agent):
  """Deploy agent to agent engine.
  Args:
    root_agent: The ADK agent to deploy.
  """
  app = vertexai.agent_engines.AdkApp(
      agent=root_agent,
  )
  remote_app = client.agent_engines.create(
      agent=app,
      config = {
          "staging_bucket": gs://BUCKET_NAME,
          "requirements": ['google-cloud-aiplatform[adk,agent_engines]'],
          "env_vars": {"GOOGLE_CLOUD_AGENT_ENGINE_ENABLE_TELEMETRY": "true"}
      }
  )
  return remote_app

agent_engine = deploy_adk_agent(my_agent)
agent_engine_resource_name = agent_engine.api_resource.name

如需获取已部署到 Agent Platform 的智能体列表，请参阅管理已部署的智能体。

生成回答

  1. 使用 run_inference() 为数据集生成模型回答：

    以 Pandas DataFrame 的形式准备数据集：提示应针对您的代理。跟踪记录需要会话输入。如需了解详情，请参阅会话：跟踪个别对话。

    import pandas as pd
from vertexai import types

session_inputs = types.evals.SessionInput(
    user_id="user_123",
    state={},
)
agent_prompts = [
    "Search for 'noise-cancelling headphones'.",
    "Show me the details for product 'B08H8H8H8H'.",
    "Add one pair of 'B08H8H8H8H' to my shopping cart.",
    "Find 'wireless earbuds' and then add the first result to my cart.",
    "I need a new laptop for work, can you find one with at least 16GB of RAM?",
]
agent_dataset = pd.DataFrame({
    "prompt": agent_prompts,
    "session_inputs": [session_inputs] * len(agent_prompts),
})

  2. 使用 run_inference() 生成模型回答：

    agent_dataset_with_inference = client.evals.run_inference(
    agent=agent_engine_resource_name,
    src=agent_dataset,
)

  3. 通过对 EvaluationDataset 对象调用 .show() 来直观呈现推理结果，以便检查模型输出以及原始提示和引用内容：

    agent_dataset_with_inference.show()

运行代理评估

运行 create_evaluation_run() 以评估代理回答。

  1. 使用内置辅助函数检索 agent_info：

    agent_info = types.evals.AgentInfo.load_from_agent(
    my_agent,
    agent_engine_resource_name
)

  2. 使用特定于智能体的基于自适应评分准则的指标（FINAL_RESPONSE_QUALITY、TOOL_USE_QUALITY 和 HALLUCINATION）评估模型回答：

    evaluation_run = client.evals.create_evaluation_run(
    dataset=agent_dataset_with_inference,
    agent_info=agent_info,
    metrics=[
        types.RubricMetric.FINAL_RESPONSE_QUALITY,
        types.RubricMetric.TOOL_USE_QUALITY,
        types.RubricMetric.HALLUCINATION,
        types.RubricMetric.SAFETY,
    ],
    dest=GCS_DEST,
)

查看代理评估结果

您可以使用 Agent Platform SDK 查看评估结果。

通过调用 .show() 检索评估运行并直观呈现评估结果，以显示摘要指标和详细结果：

evaluation_run = client.evals.get_evaluation_run(
    name=evaluation_run.name,
    include_evaluation_items=True
)

evaluation_run.show()

详细结果还包括显示代理互动的跟踪记录。如需详细了解跟踪记录，请参阅跟踪代理。

后续步骤

试用以下智能体评估笔记本：