public static final class CustomCodeExecutionSpec.Builder extends GeneratedMessage.Builder<CustomCodeExecutionSpec.Builder> implements CustomCodeExecutionSpecOrBuilderSpecificies a metric that is populated by evaluating user-defined Python code.
Protobuf type google.cloud.aiplatform.v1beta1.CustomCodeExecutionSpec
Inheritance
java.lang.Object > AbstractMessageLite.Builder<MessageType,BuilderType> > AbstractMessage.Builder<BuilderType> > GeneratedMessage.Builder > CustomCodeExecutionSpec.BuilderImplements
CustomCodeExecutionSpecOrBuilderStatic Methods
getDescriptor()
public static final Descriptors.Descriptor getDescriptor()| Returns | |
|---|---|
| Type | Description |
Descriptor |
|
Methods
build()
public CustomCodeExecutionSpec build()| Returns | |
|---|---|
| Type | Description |
CustomCodeExecutionSpec |
|
buildPartial()
public CustomCodeExecutionSpec buildPartial()| Returns | |
|---|---|
| Type | Description |
CustomCodeExecutionSpec |
|
clear()
public CustomCodeExecutionSpec.Builder clear()| Returns | |
|---|---|
| Type | Description |
CustomCodeExecutionSpec.Builder |
|
clearEvaluationFunction()
public CustomCodeExecutionSpec.Builder clearEvaluationFunction()Required. Python function. Expected user to define the following function, e.g.: def evaluate(instance: dict[str, Any]) -> float: Please include this function signature in the code snippet. Instance is the evaluation instance, any fields populated in the instance are available to the function as instance[field_name].
Example:
Example input:
`
instance= EvaluationInstance(
response=EvaluationInstance.InstanceData(text="The answer is 4."),
reference=EvaluationInstance.InstanceData(text="4")
)
Example converted input:
{
'response': {'text': 'The answer is 4.'},
'reference': {'text': '4'}
}
Example python function:
def evaluate(instance: dict[str, Any]) -> float:
if instance'response' == instance'reference':
return 1.0
return 0.0
CustomCodeExecutionSpec is also supported in Batch Evaluation (EvalDataset RPC) and Tuning Evaluation. Each line in the input jsonl file will be converted to dict[str, Any] and passed to the evaluation function.
optional string evaluation_function = 1 [(.google.api.field_behavior) = REQUIRED];
| Returns | |
|---|---|
| Type | Description |
CustomCodeExecutionSpec.Builder |
This builder for chaining. |
getDefaultInstanceForType()
public CustomCodeExecutionSpec getDefaultInstanceForType()| Returns | |
|---|---|
| Type | Description |
CustomCodeExecutionSpec |
|
getDescriptorForType()
public Descriptors.Descriptor getDescriptorForType()| Returns | |
|---|---|
| Type | Description |
Descriptor |
|
getEvaluationFunction()
public String getEvaluationFunction()Required. Python function. Expected user to define the following function, e.g.: def evaluate(instance: dict[str, Any]) -> float: Please include this function signature in the code snippet. Instance is the evaluation instance, any fields populated in the instance are available to the function as instance[field_name].
Example:
Example input:
`
instance= EvaluationInstance(
response=EvaluationInstance.InstanceData(text="The answer is 4."),
reference=EvaluationInstance.InstanceData(text="4")
)
Example converted input:
{
'response': {'text': 'The answer is 4.'},
'reference': {'text': '4'}
}
Example python function:
def evaluate(instance: dict[str, Any]) -> float:
if instance'response' == instance'reference':
return 1.0
return 0.0
CustomCodeExecutionSpec is also supported in Batch Evaluation (EvalDataset RPC) and Tuning Evaluation. Each line in the input jsonl file will be converted to dict[str, Any] and passed to the evaluation function.
optional string evaluation_function = 1 [(.google.api.field_behavior) = REQUIRED];
| Returns | |
|---|---|
| Type | Description |
String |
The evaluationFunction. |
getEvaluationFunctionBytes()
public ByteString getEvaluationFunctionBytes()Required. Python function. Expected user to define the following function, e.g.: def evaluate(instance: dict[str, Any]) -> float: Please include this function signature in the code snippet. Instance is the evaluation instance, any fields populated in the instance are available to the function as instance[field_name].
Example:
Example input:
`
instance= EvaluationInstance(
response=EvaluationInstance.InstanceData(text="The answer is 4."),
reference=EvaluationInstance.InstanceData(text="4")
)
Example converted input:
{
'response': {'text': 'The answer is 4.'},
'reference': {'text': '4'}
}
Example python function:
def evaluate(instance: dict[str, Any]) -> float:
if instance'response' == instance'reference':
return 1.0
return 0.0
CustomCodeExecutionSpec is also supported in Batch Evaluation (EvalDataset RPC) and Tuning Evaluation. Each line in the input jsonl file will be converted to dict[str, Any] and passed to the evaluation function.
optional string evaluation_function = 1 [(.google.api.field_behavior) = REQUIRED];
| Returns | |
|---|---|
| Type | Description |
ByteString |
The bytes for evaluationFunction. |
hasEvaluationFunction()
public boolean hasEvaluationFunction()Required. Python function. Expected user to define the following function, e.g.: def evaluate(instance: dict[str, Any]) -> float: Please include this function signature in the code snippet. Instance is the evaluation instance, any fields populated in the instance are available to the function as instance[field_name].
Example:
Example input:
`
instance= EvaluationInstance(
response=EvaluationInstance.InstanceData(text="The answer is 4."),
reference=EvaluationInstance.InstanceData(text="4")
)
Example converted input:
{
'response': {'text': 'The answer is 4.'},
'reference': {'text': '4'}
}
Example python function:
def evaluate(instance: dict[str, Any]) -> float:
if instance'response' == instance'reference':
return 1.0
return 0.0
CustomCodeExecutionSpec is also supported in Batch Evaluation (EvalDataset RPC) and Tuning Evaluation. Each line in the input jsonl file will be converted to dict[str, Any] and passed to the evaluation function.
optional string evaluation_function = 1 [(.google.api.field_behavior) = REQUIRED];
| Returns | |
|---|---|
| Type | Description |
boolean |
Whether the evaluationFunction field is set. |
internalGetFieldAccessorTable()
protected GeneratedMessage.FieldAccessorTable internalGetFieldAccessorTable()| Returns | |
|---|---|
| Type | Description |
FieldAccessorTable |
|
isInitialized()
public final boolean isInitialized()| Returns | |
|---|---|
| Type | Description |
boolean |
|
mergeFrom(CustomCodeExecutionSpec other)
public CustomCodeExecutionSpec.Builder mergeFrom(CustomCodeExecutionSpec other)| Parameter | |
|---|---|
| Name | Description |
other |
CustomCodeExecutionSpec |
| Returns | |
|---|---|
| Type | Description |
CustomCodeExecutionSpec.Builder |
|
mergeFrom(CodedInputStream input, ExtensionRegistryLite extensionRegistry)
public CustomCodeExecutionSpec.Builder mergeFrom(CodedInputStream input, ExtensionRegistryLite extensionRegistry)| Parameters | |
|---|---|
| Name | Description |
input |
CodedInputStream |
extensionRegistry |
ExtensionRegistryLite |
| Returns | |
|---|---|
| Type | Description |
CustomCodeExecutionSpec.Builder |
|
| Exceptions | |
|---|---|
| Type | Description |
IOException |
|
mergeFrom(Message other)
public CustomCodeExecutionSpec.Builder mergeFrom(Message other)| Parameter | |
|---|---|
| Name | Description |
other |
Message |
| Returns | |
|---|---|
| Type | Description |
CustomCodeExecutionSpec.Builder |
|
setEvaluationFunction(String value)
public CustomCodeExecutionSpec.Builder setEvaluationFunction(String value)Required. Python function. Expected user to define the following function, e.g.: def evaluate(instance: dict[str, Any]) -> float: Please include this function signature in the code snippet. Instance is the evaluation instance, any fields populated in the instance are available to the function as instance[field_name].
Example:
Example input:
`
instance= EvaluationInstance(
response=EvaluationInstance.InstanceData(text="The answer is 4."),
reference=EvaluationInstance.InstanceData(text="4")
)
Example converted input:
{
'response': {'text': 'The answer is 4.'},
'reference': {'text': '4'}
}
Example python function:
def evaluate(instance: dict[str, Any]) -> float:
if instance'response' == instance'reference':
return 1.0
return 0.0
CustomCodeExecutionSpec is also supported in Batch Evaluation (EvalDataset RPC) and Tuning Evaluation. Each line in the input jsonl file will be converted to dict[str, Any] and passed to the evaluation function.
optional string evaluation_function = 1 [(.google.api.field_behavior) = REQUIRED];
| Parameter | |
|---|---|
| Name | Description |
value |
StringThe evaluationFunction to set. |
| Returns | |
|---|---|
| Type | Description |
CustomCodeExecutionSpec.Builder |
This builder for chaining. |
setEvaluationFunctionBytes(ByteString value)
public CustomCodeExecutionSpec.Builder setEvaluationFunctionBytes(ByteString value)Required. Python function. Expected user to define the following function, e.g.: def evaluate(instance: dict[str, Any]) -> float: Please include this function signature in the code snippet. Instance is the evaluation instance, any fields populated in the instance are available to the function as instance[field_name].
Example:
Example input:
`
instance= EvaluationInstance(
response=EvaluationInstance.InstanceData(text="The answer is 4."),
reference=EvaluationInstance.InstanceData(text="4")
)
Example converted input:
{
'response': {'text': 'The answer is 4.'},
'reference': {'text': '4'}
}
Example python function:
def evaluate(instance: dict[str, Any]) -> float:
if instance'response' == instance'reference':
return 1.0
return 0.0
CustomCodeExecutionSpec is also supported in Batch Evaluation (EvalDataset RPC) and Tuning Evaluation. Each line in the input jsonl file will be converted to dict[str, Any] and passed to the evaluation function.
optional string evaluation_function = 1 [(.google.api.field_behavior) = REQUIRED];
| Parameter | |
|---|---|
| Name | Description |
value |
ByteStringThe bytes for evaluationFunction to set. |
| Returns | |
|---|---|
| Type | Description |
CustomCodeExecutionSpec.Builder |
This builder for chaining. |