REST Resource: projects.locations.brands.evaluationCases

Resource: EvaluationCase

An individual evaluation case.

JSON representation 
{
  "name": string,
  "displayName": string,
  "description": string,
  "evaluationContext": string,
  "evaluationMatchers": [
    string
  ],

  // Union field test_case can be only one of the following:
  "staticTextTurnCase": {
    object (StaticTextTurnCase)
  },
  "dynamicTextTurnCase": {
    object (DynamicTextTurnCase)
  }
  // End of list of possible types for union field test_case.
}
Fields
name

string

Identifier. The resource name of the evaluation case. Format: projects/{project}/locations/{location}/brands/{brand}/evaluationCases/{evaluationCase}
displayName

string

Required. The human-readable name of the evaluation case.
description

string

Optional. The description of the evaluation case.
evaluationContext

string

Required. The resource name of the evaluation context this case is associated with. Format: projects/{project}/locations/{location}/brands/{brand}/evaluationContexts/{evaluationContext}
evaluationMatchers[]

string

Optional. The list of evaluation matchers to use for the evaluation case. The evaluation is performed once at the end of the BidiProcessOrder session. Each matcher produces a boolean result indicating whether the evaluation matcher passed or failed.
Union field test_case. The test case to use for the evaluation case. test_case can be only one of the following:
staticTextTurnCase

object (StaticTextTurnCase)

Optional. A test case where the user and agent text turns are static.
dynamicTextTurnCase

object (DynamicTextTurnCase)

Optional. A test case where the user text inputs are dynamically generated using LLM with an ordering instruction.

StaticTextTurnCase

An evaluation case where the user inputs are static text inputs.

JSON representation 
{
  "textTurns": [
    {
      object (TextTurn)
    }
  ]
}
Fields
textTurns[]

object (TextTurn)

Required. The list of text turns in the evaluation case.

TextTurn

A single text turn in the evaluation case, consisting of user input and agent response.

JSON representation 
{
  "userTextInput": string,
  "exampleAgentResponse": string
}
Fields
userTextInput

string

Required. The user text input for this turn.
exampleAgentResponse

string

Optional. The example agent response for this turn. Note that this is not the actual agent response. The actual agent response may differ from this example response. The example response is NOT used for evaluation, but only for display purposes to make the evaluation case easier to understand.

DynamicTextTurnCase

An evaluation case where the user inputs are dynamically generated using LLM with an ordering instruction.

JSON representation 
{
  "orderingInstruction": string
}
Fields
orderingInstruction

string

Required. The ordering instruction for the user to interact with the Food Ordering Agent. LLM dynamically generates the user text inputs based on the ordering instruction. This may include the targeted items to order, the user's preferences and constraints, and specific actions to that the user should inform the ordering agent.

Methods

batchCreate

 Batch creates EvaluationCases.

batchUpdate

 Batch updates EvaluationCases.

create

 Creates an EvaluationCase.

delete

 Deletes the specified EvaluationCase.

get

 Gets an EvaluationCase.

list

 Lists EvaluationCases.

patch

 Updates the specified EvaluationCase.

run

 Runs an evaluation case.