Vertex AI v1beta1 API - Class CustomCodeExecutionSpec (1.0.0-beta77) 

public sealed class CustomCodeExecutionSpec : IMessage<CustomCodeExecutionSpec>, IEquatable<CustomCodeExecutionSpec>, IDeepCloneable<CustomCodeExecutionSpec>, IBufferMessage, IMessage

Reference documentation and code samples for the Vertex AI v1beta1 API class CustomCodeExecutionSpec.

Specificies a metric that is populated by evaluating user-defined Python code.

Inheritance

object > CustomCodeExecutionSpec

Namespace

Google.Cloud.AIPlatform.V1Beta1

Assembly

Google.Cloud.AIPlatform.V1Beta1.dll

Constructors

CustomCodeExecutionSpec()

public CustomCodeExecutionSpec()

CustomCodeExecutionSpec(CustomCodeExecutionSpec)

public CustomCodeExecutionSpec(CustomCodeExecutionSpec other)
Parameter
Name Description
other CustomCodeExecutionSpec

Properties

EvaluationFunction

public string EvaluationFunction { get; set; }

Required. Python function. Expected user to define the following function, e.g.: def evaluate(instance: dict[str, Any]) -> float: Please include this function signature in the code snippet. Instance is the evaluation instance, any fields populated in the instance are available to the function as instance[field_name].

Example: Example input:

instance= EvaluationInstance(
    response=EvaluationInstance.InstanceData(text="The answer is 4."),
    reference=EvaluationInstance.InstanceData(text="4")
)

Example converted input:

{
 'response': {'text': 'The answer is 4.'},
 'reference': {'text': '4'}
}

Example python function:

 def evaluate(instance: dict[str, Any]) -> float:
   if instance['response']['text'] == instance['reference']['text']:
     return 1.0
   return 0.0

CustomCodeExecutionSpec is also supported in Batch Evaluation (EvalDataset RPC) and Tuning Evaluation. Each line in the input jsonl file will be converted to dict[str, Any] and passed to the evaluation function.

Property Value
Type Description
string

HasEvaluationFunction

public bool HasEvaluationFunction { get; }

Gets whether the "evaluation_function" field is set

Property Value
Type Description
bool