Tutorial: Melakukan evaluasi menggunakan Klien GenAI di Agent Platform SDK

Untuk melihat contoh Memulai: Melakukan evaluasi menggunakan GenAI Client, jalankan notebook "Memulai: Melakukan evaluasi menggunakan GenAI Client" di salah satu lingkungan berikut:

Buka di Colab | Buka di Colab Enterprise | Buka di Agent Platform Workbench | Lihat di GitHub

Halaman ini menunjukkan cara mengevaluasi model dan aplikasi AI generatif Anda di berbagai kasus penggunaan menggunakan GenAI Client di Agent Platform SDK.

Sebelum memulai

1. Login keakun Anda. Google Cloud Jika Anda baru menggunakan Google Cloud, buat akun untuk mengevaluasi performa produk kami dalam skenario dunia nyata. Pelanggan baru juga mendapatkan kredit gratis senilai $300 untuk menjalankan, menguji, dan men-deploy workload.

   In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

   Roles required to select or create a project

   • Select a project: Selecting a project doesn't require a specific IAM role—you can select any project that you've been granted a role on.
   • Create a project: To create a project, you need the Project Creator role (roles/resourcemanager.projectCreator), which contains the resourcemanager.projects.create permission. Learn how to grant roles.

   Note: If you don't plan to keep the resources that you create in this procedure, create a project instead of selecting an existing project. After you finish these steps, you can delete the project, removing all resources associated with the project.

   Go to project selector

   Verify that billing is enabled for your Google Cloud project.

   In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

   Roles required to select or create a project

   • Select a project: Selecting a project doesn't require a specific IAM role—you can select any project that you've been granted a role on.
   • Create a project: To create a project, you need the Project Creator role (roles/resourcemanager.projectCreator), which contains the resourcemanager.projects.create permission. Learn how to grant roles.

   Note: If you don't plan to keep the resources that you create in this procedure, create a project instead of selecting an existing project. After you finish these steps, you can delete the project, removing all resources associated with the project.

   Go to project selector

   Verify that billing is enabled for your Google Cloud project.

2. Instal Agent Platform SDK:

   !pip install google-cloud-aiplatform[evaluation]
   

3. Siapkan kredensial Anda. Jika Anda menjalankan tutorial ini di Colaboratory, jalankan perintah berikut:

   from google.colab import auth
   auth.authenticate_user()
   

   Untuk lingkungan lainnya, lihat Mengautentikasi ke Agent Platform.

Menginisialisasi GenAI Client

Untuk menginisialisasi GenAI Client, jalankan perintah berikut:

from vertexai import Client

client = Client(project="YOUR_PROJECT_ID", location="YOUR_LOCATION")

Dengan:

• YOUR_PROJECT_ID: ID project Anda. Google Cloud
• YOUR_LOCATION: region cloud Anda, misalnya, us-central1.

Membuat respons

Buat respons model untuk set data Anda menggunakan run_inference():

1. Siapkan set data Anda sebagai Pandas DataFrame:

   import pandas as pd
   
   eval_df = pd.DataFrame({
     "prompt": [
         "Explain software 'technical debt' using a concise analogy of planting a garden.",
         "Write a Python function to find the nth Fibonacci number using recursion with memoization, but without using any imports.",
         "Write a four-line poem about a lonely robot, where every line must be a question and the word 'and' cannot be used.",
         "A drawer has 10 red socks and 10 blue socks. In complete darkness, what is the minimum number of socks you must pull out to guarantee you have a matching pair?",
         "An AI discovers a cure for a major disease, but the cure is based on private data it analyzed without consent. Should the cure be released? Justify your answer."
     ]
   })
   

2. Buat respons model menggunakan run_inference():

   eval_dataset = client.evals.run_inference(
     model="gemini-2.5-flash",
     src=eval_df,
   )
   

3. Visualisasikan hasil inferensi Anda dengan memanggil .show() pada objek EvaluationDataset untuk memeriksa output model bersama dengan perintah dan referensi asli Anda:

   eval_dataset.show()
   

Gambar berikut menampilkan set data evaluasi dengan perintah dan respons yang sesuai:

Menjalankan evaluasi

Jalankan evaluate() untuk mengevaluasi respons model:

1. Evaluasi respons model menggunakan metrik berbasis rubrik adaptif GENERAL_QUALITY default:

   eval_result = client.evals.evaluate(dataset=eval_dataset)
   

2. Visualisasikan hasil evaluasi Anda dengan memanggil .show() pada objek EvaluationResult untuk menampilkan metrik ringkasan dan hasil mendetail:

   eval_result.show()
   

Gambar berikut menampilkan laporan evaluasi, yang menunjukkan metrik ringkasan dan hasil mendetail untuk setiap pasangan perintah-respons.

Pembersihan

Tidak ada resource Gemini Enterprise Agent Platform yang dibuat selama tutorial ini.

Langkah berikutnya

• Tentukan metrik evaluasi Anda.