Class EvaluateInstancesRequest (1.158.0)

1.158.0 (latest) 1.157.0 1.156.0 1.155.0 1.154.0 1.153.1 1.152.0 1.151.0 1.150.0 1.149.0 1.148.1 1.147.0 1.146.0 1.145.0 1.144.0 1.143.0 1.142.0 1.141.0 1.140.0 1.139.0 1.138.0 1.137.0 1.136.0 1.135.0 1.134.0 1.133.0 1.132.0 1.131.0 1.130.0 1.129.0 1.122.0 1.121.0 1.120.0 1.119.0 1.118.0 1.117.0 1.116.0 1.115.0 1.114.0 1.113.0 1.112.0 1.111.0 1.110.0 1.109.0 1.108.0 1.107.0 1.106.0 1.105.0 1.104.0 1.103.0 1.102.0 1.101.0 1.100.0 1.99.0 1.98.0 1.97.0 1.96.0 1.95.1 1.94.0 1.93.1 1.92.0 1.91.0 1.90.0 1.89.0 1.88.0 1.87.0 1.86.0 1.85.0 1.84.0 1.83.0 1.82.0 1.81.0 1.80.0 1.79.0 1.78.0 1.77.0 1.76.0 1.75.0 1.74.0 1.73.0 1.72.0 1.71.1 1.70.0 1.69.0 1.68.0 1.67.1 1.66.0 1.65.0 1.63.0 1.62.0 1.60.0 1.59.0 1.58.0 1.57.0 1.56.0 1.55.0 1.54.1 1.53.0 1.52.0 1.51.0 1.50.0 1.49.0 1.48.0 1.47.0 1.46.0 1.45.0 1.44.0 1.43.0 1.39.0 1.38.1 1.37.0 1.36.4 1.35.0 1.34.0 1.33.1 1.32.0 1.31.1 1.30.1 1.29.0 1.28.1 1.27.1 1.26.1 1.25.0 1.24.1 1.23.0 1.22.1 1.21.0 1.20.0 1.19.1 1.18.3 1.17.1 1.16.1 1.15.1 1.14.0 1.13.1 1.12.1 1.11.0 1.10.0 1.9.0 1.8.1 1.7.1 1.6.2 1.5.0 1.4.3 1.3.0 1.2.0 1.1.1 1.0.1 0.9.0 0.8.0 0.7.1 0.6.0 0.5.1 0.4.0 0.3.1

EvaluateInstancesRequest(mapping=None, *, ignore_unknown_fields=False, **kwargs)

Request message for EvaluationService.EvaluateInstances.

This message has oneof_ fields (mutually exclusive fields). For each oneof, at most one member field can be set at the same time. Setting any member of the oneof automatically clears all other members.

.. _oneof: https://proto-plus-python.readthedocs.io/en/stable/fields.html#oneofs-mutually-exclusive-fields

Attributes
Name	Description
exact_match_input	google.cloud.aiplatform_v1beta1.types.ExactMatchInput Auto metric instances. Instances and metric spec for exact match metric. This field is a member of oneof_ metric_inputs.
bleu_input	google.cloud.aiplatform_v1beta1.types.BleuInput Instances and metric spec for bleu metric. This field is a member of oneof_ metric_inputs.
rouge_input	google.cloud.aiplatform_v1beta1.types.RougeInput Instances and metric spec for rouge metric. This field is a member of oneof_ metric_inputs.
fluency_input	google.cloud.aiplatform_v1beta1.types.FluencyInput LLM-based metric instance. General text generation metrics, applicable to other categories. Input for fluency metric. This field is a member of oneof_ metric_inputs.
coherence_input	google.cloud.aiplatform_v1beta1.types.CoherenceInput Input for coherence metric. This field is a member of oneof_ metric_inputs.
safety_input	google.cloud.aiplatform_v1beta1.types.SafetyInput Input for safety metric. This field is a member of oneof_ metric_inputs.
groundedness_input	google.cloud.aiplatform_v1beta1.types.GroundednessInput Input for groundedness metric. This field is a member of oneof_ metric_inputs.
fulfillment_input	google.cloud.aiplatform_v1beta1.types.FulfillmentInput Input for fulfillment metric. This field is a member of oneof_ metric_inputs.
summarization_quality_input	google.cloud.aiplatform_v1beta1.types.SummarizationQualityInput Input for summarization quality metric. This field is a member of oneof_ metric_inputs.
pairwise_summarization_quality_input	google.cloud.aiplatform_v1beta1.types.PairwiseSummarizationQualityInput Input for pairwise summarization quality metric. This field is a member of oneof_ metric_inputs.
summarization_helpfulness_input	google.cloud.aiplatform_v1beta1.types.SummarizationHelpfulnessInput Input for summarization helpfulness metric. This field is a member of oneof_ metric_inputs.
summarization_verbosity_input	google.cloud.aiplatform_v1beta1.types.SummarizationVerbosityInput Input for summarization verbosity metric. This field is a member of oneof_ metric_inputs.
question_answering_quality_input	google.cloud.aiplatform_v1beta1.types.QuestionAnsweringQualityInput Input for question answering quality metric. This field is a member of oneof_ metric_inputs.
pairwise_question_answering_quality_input	google.cloud.aiplatform_v1beta1.types.PairwiseQuestionAnsweringQualityInput Input for pairwise question answering quality metric. This field is a member of oneof_ metric_inputs.
question_answering_relevance_input	google.cloud.aiplatform_v1beta1.types.QuestionAnsweringRelevanceInput Input for question answering relevance metric. This field is a member of oneof_ metric_inputs.
question_answering_helpfulness_input	google.cloud.aiplatform_v1beta1.types.QuestionAnsweringHelpfulnessInput Input for question answering helpfulness metric. This field is a member of oneof_ metric_inputs.
question_answering_correctness_input	google.cloud.aiplatform_v1beta1.types.QuestionAnsweringCorrectnessInput Input for question answering correctness metric. This field is a member of oneof_ metric_inputs.
pointwise_metric_input	google.cloud.aiplatform_v1beta1.types.PointwiseMetricInput Input for pointwise metric. This field is a member of oneof_ metric_inputs.
pairwise_metric_input	google.cloud.aiplatform_v1beta1.types.PairwiseMetricInput Input for pairwise metric. This field is a member of oneof_ metric_inputs.
tool_call_valid_input	google.cloud.aiplatform_v1beta1.types.ToolCallValidInput Tool call metric instances. Input for tool call valid metric. This field is a member of oneof_ metric_inputs.
tool_name_match_input	google.cloud.aiplatform_v1beta1.types.ToolNameMatchInput Input for tool name match metric. This field is a member of oneof_ metric_inputs.
tool_parameter_key_match_input	google.cloud.aiplatform_v1beta1.types.ToolParameterKeyMatchInput Input for tool parameter key match metric. This field is a member of oneof_ metric_inputs.
tool_parameter_kv_match_input	google.cloud.aiplatform_v1beta1.types.ToolParameterKVMatchInput Input for tool parameter key value match metric. This field is a member of oneof_ metric_inputs.
comet_input	google.cloud.aiplatform_v1beta1.types.CometInput Translation metrics. Input for Comet metric. This field is a member of oneof_ metric_inputs.
metricx_input	google.cloud.aiplatform_v1beta1.types.MetricxInput Input for Metricx metric. This field is a member of oneof_ metric_inputs.
trajectory_exact_match_input	google.cloud.aiplatform_v1beta1.types.TrajectoryExactMatchInput Input for trajectory exact match metric. This field is a member of oneof_ metric_inputs.
trajectory_in_order_match_input	google.cloud.aiplatform_v1beta1.types.TrajectoryInOrderMatchInput Input for trajectory in order match metric. This field is a member of oneof_ metric_inputs.
trajectory_any_order_match_input	google.cloud.aiplatform_v1beta1.types.TrajectoryAnyOrderMatchInput Input for trajectory match any order metric. This field is a member of oneof_ metric_inputs.
trajectory_precision_input	google.cloud.aiplatform_v1beta1.types.TrajectoryPrecisionInput Input for trajectory precision metric. This field is a member of oneof_ metric_inputs.
trajectory_recall_input	google.cloud.aiplatform_v1beta1.types.TrajectoryRecallInput Input for trajectory recall metric. This field is a member of oneof_ metric_inputs.
trajectory_single_tool_use_input	google.cloud.aiplatform_v1beta1.types.TrajectorySingleToolUseInput Input for trajectory single tool use metric. This field is a member of oneof_ metric_inputs.
rubric_based_instruction_following_input	google.cloud.aiplatform_v1beta1.types.RubricBasedInstructionFollowingInput Rubric Based Instruction Following metric. This field is a member of oneof_ metric_inputs.
location	str Required. The resource name of the Location to evaluate the instances. Format: projects/{project}/locations/{location}
metrics	MutableSequence[google.cloud.aiplatform_v1beta1.types.Metric] The metrics used for evaluation. Currently, we only support evaluating a single metric. If multiple metrics are provided, only the first one will be evaluated.
metric_sources	MutableSequence[google.cloud.aiplatform_v1beta1.types.MetricSource] Optional. The metrics (either inline or registered) used for evaluation. Currently, we only support evaluating a single metric. If multiple metrics are provided, only the first one will be evaluated.
instance	google.cloud.aiplatform_v1beta1.types.EvaluationInstance The instance to be evaluated.
autorater_config	google.cloud.aiplatform_v1beta1.types.AutoraterConfig Optional. Autorater config used for evaluation.

Methods

EvaluateInstancesRequest

EvaluateInstancesRequest(mapping=None, *, ignore_unknown_fields=False, **kwargs)

Request message for EvaluationService.EvaluateInstances.

This message has oneof_ fields (mutually exclusive fields). For each oneof, at most one member field can be set at the same time. Setting any member of the oneof automatically clears all other members.

.. _oneof: https://proto-plus-python.readthedocs.io/en/stable/fields.html#oneofs-mutually-exclusive-fields