EvaluationMetricsConfig

Configures the metrics for an evaluation.

JSON representation 
{
  "goldenMetricsConfig": {
    object (EvaluationMetricsConfig.GoldenMetricsConfig)
  },
  "scenarioMetricsConfig": {
    object (EvaluationMetricsConfig.ScenarioMetricsConfig)
  }
}
Fields
goldenMetricsConfig

object (EvaluationMetricsConfig.GoldenMetricsConfig)

Optional. Configuration for the golden metrics for the evaluation.
scenarioMetricsConfig

object (EvaluationMetricsConfig.ScenarioMetricsConfig)

Optional. Configuration for the scenario metrics for the evaluation.

EvaluationMetricsConfig.GoldenMetricsConfig

Configuration for the golden metrics for the evaluation.

JSON representation 
{
  "semanticSimilarityMetricsConfig": {
    object (EvaluationMetricsConfig.SemanticSimilarityMetricsConfig)
  },
  "toolCorrectnessMetricsConfig": {
    object (EvaluationMetricsConfig.ToolCorrectnessMetricsConfig)
  },
  "stepToolCorrectnessMetricsConfig": {
    object (EvaluationMetricsConfig.ToolCorrectnessMetricsConfig)
  }
}
Fields
semanticSimilarityMetricsConfig

object (EvaluationMetricsConfig.SemanticSimilarityMetricsConfig)

Optional. Global configuration for semantic similarity metrics.
toolCorrectnessMetricsConfig

object (EvaluationMetricsConfig.ToolCorrectnessMetricsConfig)

Optional. Configuration for turn level tool correctness metrics.
stepToolCorrectnessMetricsConfig

object (EvaluationMetricsConfig.ToolCorrectnessMetricsConfig)

Optional. Configuration for step level tool correctness metrics.

EvaluationMetricsConfig.ToolCorrectnessMetricsConfig

Configuration for correctness metrics for the evaluation. To disable the metric, set the message but do not set the enableToolCorrectnessMetrics field to true (or explicitly set it to false). To unset the configuration and fallback to the default behavior, omit the message entirely.

JSON representation 
{
  "enableToolCorrectnessMetrics": boolean
}
Fields
enableToolCorrectnessMetrics

boolean

Optional. Whether to calculate tool correctness metrics for the evaluation.

EvaluationMetricsConfig.ScenarioMetricsConfig

Configuration for the scenario metrics for the evaluation.

JSON representation 
{
  "userGoalMetMetricsConfig": {
    object (EvaluationMetricsConfig.UserGoalMetMetricsConfig)
  },
  "expectationsMetMetricsConfig": {
    object (EvaluationMetricsConfig.ExpectationsMetMetricsConfig)
  }
}
Fields
userGoalMetMetricsConfig

object (EvaluationMetricsConfig.UserGoalMetMetricsConfig)

Optional. Configuration for user goal met metrics.
expectationsMetMetricsConfig

object (EvaluationMetricsConfig.ExpectationsMetMetricsConfig)

Optional. Configuration for expectation level metrics.

EvaluationMetricsConfig.UserGoalMetMetricsConfig

Configuration for the user goal met metrics for the evaluation. To disable the metric, set the message but do not set the enableUserGoalMetMetrics field to true (or explicitly set it to false). To unset the configuration and fallback to the default behavior, omit the message entirely.

JSON representation 
{
  "enableUserGoalMetMetrics": boolean
}
Fields
enableUserGoalMetMetrics

boolean

Optional. Whether to calculate the user goal met metrics for the evaluation.

EvaluationMetricsConfig.ExpectationsMetMetricsConfig

Configuration for the expectation level metrics for the evaluation. To disable the metric, set the message but do not set the enableExpectationsMetMetrics field to true (or explicitly set it to false). To unset the configuration and fallback to the default behavior, omit the message entirely.

JSON representation 
{
  "enableExpectationsMetMetrics": boolean
}
Fields
enableExpectationsMetMetrics

boolean

Optional. Whether to calculate the expectation level metrics for the evaluation.