Class EvaluationDataset (2.4.0) 

EvaluationDataset(**data: typing.Any)

The dataset used for evaluation.

Methods

EvaluationDataset

EvaluationDataset(**data: typing.Any)

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

load_from_adk_eval_set

load_from_adk_eval_set(
    eval_set: typing.Union[str, os.PathLike[str], dict[str, typing.Any]],
) -> vertexai._genai.types.common.EvaluationDataset

Loads an ADK eval set (.evalset.json) into an EvaluationDataset.

Each ADK eval case becomes an EvalCase with eval_case_id set to its eval_id. For a static conversation, the last invocation's user content becomes prompt and its final response becomes reference. Earlier invocations, including their tool calls, become conversation_history. A conversation scenario becomes user_scenario. Rubrics on the case and on the last invocation go into rubric_groups, keyed by rubric type ("adk_rubrics" when unset).

Some ADK data is kept as EvalCase extra fields: reference_trajectory holds the tool calls of the last invocation, session_inputs holds the session input, and custom ADK eval case fields keep their names. Other ADK fields, such as final_session_state and the scenario's user_persona, are not carried over. Steps that convert eval cases to a DataFrame, such as run_inference, drop the extra fields, rubric_groups and eval_case_id. google-adk does not need to be installed.

.. rubric:: Example

import vertexai from vertexai._genai import types

client = vertexai.Client( project="my-project", location="us-central1" ) dataset = types.EvaluationDataset.load_from_adk_eval_set( "my_agent/my_eval_set.evalset.json" ) dataset_with_responses = client.evals.run_inference( src=dataset, agent=my_agent ) result = client.evals.evaluate( dataset=dataset_with_responses, metrics=[types.RubricMetric.FINAL_RESPONSE_MATCH], )

load_from_observability_eval_cases

load_from_observability_eval_cases(
    cases: list["ObservabilityEvalCase"],
) -> vertexai._genai.types.common.EvaluationDataset

Fetches GenAI Observability data from GCS and parses into a DataFrame.

show

show() -> None

Shows the evaluation dataset.