GKE Recommender v1 API - Class GkeInferenceQuickstart.GkeInferenceQuickstartClient (1.0.0-beta01)

public class GkeInferenceQuickstart.GkeInferenceQuickstartClient : ClientBase<GkeInferenceQuickstart.GkeInferenceQuickstartClient>

Reference documentation and code samples for the GKE Recommender v1 API class GkeInferenceQuickstart.GkeInferenceQuickstartClient.

Client for GkeInferenceQuickstart

Inheritance

object > ClientBase > ClientBaseGkeInferenceQuickstartGkeInferenceQuickstartClient > GkeInferenceQuickstart.GkeInferenceQuickstartClient

Inherited Members

ClientBase<GkeInferenceQuickstart.GkeInferenceQuickstartClient>.WithHost(string)

ClientBase.CallInvoker

object.GetHashCode()

object.GetType()

object.MemberwiseClone()

object.ToString()

Namespace

Google.Cloud.GkeRecommender.V1

Assembly

Google.Cloud.GkeRecommender.V1.dll

Constructors

GkeInferenceQuickstartClient()

protected GkeInferenceQuickstartClient()

Protected parameterless constructor to allow creation of test doubles.

GkeInferenceQuickstartClient(CallInvoker)

public GkeInferenceQuickstartClient(CallInvoker callInvoker)

Creates a new client for GkeInferenceQuickstart that uses a custom CallInvoker.

Parameter
Name	Description
callInvoker	CallInvoker The callInvoker to use to make remote calls.

GkeInferenceQuickstartClient(ChannelBase)

public GkeInferenceQuickstartClient(ChannelBase channel)

Creates a new client for GkeInferenceQuickstart

Parameter
Name	Description
channel	ChannelBase The channel to use to make remote calls.

GkeInferenceQuickstartClient(ClientBaseConfiguration)

protected GkeInferenceQuickstartClient(ClientBase.ClientBaseConfiguration configuration)

Protected constructor to allow creation of configured clients.

Parameter
Name	Description
configuration	ClientBaseClientBaseConfiguration The client configuration.

Methods

FetchBenchmarkingData(FetchBenchmarkingDataRequest, CallOptions)

public virtual FetchBenchmarkingDataResponse FetchBenchmarkingData(FetchBenchmarkingDataRequest request, CallOptions options)

Fetches all of the benchmarking data available for a profile. Benchmarking data returns all of the performance metrics available for a given model server setup on a given instance type.

Parameters
Name	Description
request	FetchBenchmarkingDataRequest The request to send to the server.
options	CallOptions The options for the call.

Returns
Type	Description
FetchBenchmarkingDataResponse	The response received from the server.

FetchBenchmarkingData(FetchBenchmarkingDataRequest, Metadata, DateTime?, CancellationToken)

public virtual FetchBenchmarkingDataResponse FetchBenchmarkingData(FetchBenchmarkingDataRequest request, Metadata headers = null, DateTime? deadline = null, CancellationToken cancellationToken = default)

Fetches all of the benchmarking data available for a profile. Benchmarking data returns all of the performance metrics available for a given model server setup on a given instance type.

Parameters
Name	Description
request	FetchBenchmarkingDataRequest The request to send to the server.
headers	Metadata The initial metadata to send with the call. This parameter is optional.
deadline	System.DateTime An optional deadline for the call. The call will be cancelled if deadline is hit.
cancellationToken	CancellationToken An optional token for canceling the call.

Returns
Type	Description
FetchBenchmarkingDataResponse	The response received from the server.

FetchBenchmarkingDataAsync(FetchBenchmarkingDataRequest, CallOptions)

public virtual AsyncUnaryCall<FetchBenchmarkingDataResponse> FetchBenchmarkingDataAsync(FetchBenchmarkingDataRequest request, CallOptions options)

Fetches all of the benchmarking data available for a profile. Benchmarking data returns all of the performance metrics available for a given model server setup on a given instance type.

Parameters
Name	Description
request	FetchBenchmarkingDataRequest The request to send to the server.
options	CallOptions The options for the call.

Returns
Type	Description
AsyncUnaryCallFetchBenchmarkingDataResponse	The call object.

FetchBenchmarkingDataAsync(FetchBenchmarkingDataRequest, Metadata, DateTime?, CancellationToken)

public virtual AsyncUnaryCall<FetchBenchmarkingDataResponse> FetchBenchmarkingDataAsync(FetchBenchmarkingDataRequest request, Metadata headers = null, DateTime? deadline = null, CancellationToken cancellationToken = default)

Fetches all of the benchmarking data available for a profile. Benchmarking data returns all of the performance metrics available for a given model server setup on a given instance type.

Parameters
Name	Description
request	FetchBenchmarkingDataRequest The request to send to the server.
headers	Metadata The initial metadata to send with the call. This parameter is optional.
deadline	System.DateTime An optional deadline for the call. The call will be cancelled if deadline is hit.
cancellationToken	CancellationToken An optional token for canceling the call.

Returns
Type	Description
AsyncUnaryCallFetchBenchmarkingDataResponse	The call object.

FetchModelServerVersions(FetchModelServerVersionsRequest, CallOptions)

public virtual FetchModelServerVersionsResponse FetchModelServerVersions(FetchModelServerVersionsRequest request, CallOptions options)

Fetches available model server versions. Open-source servers use their own versioning schemas (e.g., vllm uses semver like v1.0.0).

Some model servers have different versioning schemas depending on the accelerator. For example, vllm uses semver on GPUs, but returns nightly build tags on TPUs. All available versions will be returned when different schemas are present.

Parameters
Name	Description
request	FetchModelServerVersionsRequest The request to send to the server.
options	CallOptions The options for the call.

Returns
Type	Description
FetchModelServerVersionsResponse	The response received from the server.

FetchModelServerVersions(FetchModelServerVersionsRequest, Metadata, DateTime?, CancellationToken)

public virtual FetchModelServerVersionsResponse FetchModelServerVersions(FetchModelServerVersionsRequest request, Metadata headers = null, DateTime? deadline = null, CancellationToken cancellationToken = default)

Fetches available model server versions. Open-source servers use their own versioning schemas (e.g., vllm uses semver like v1.0.0).

Some model servers have different versioning schemas depending on the accelerator. For example, vllm uses semver on GPUs, but returns nightly build tags on TPUs. All available versions will be returned when different schemas are present.

Parameters
Name	Description
request	FetchModelServerVersionsRequest The request to send to the server.
headers	Metadata The initial metadata to send with the call. This parameter is optional.
deadline	System.DateTime An optional deadline for the call. The call will be cancelled if deadline is hit.
cancellationToken	CancellationToken An optional token for canceling the call.

Returns
Type	Description
FetchModelServerVersionsResponse	The response received from the server.

FetchModelServerVersionsAsync(FetchModelServerVersionsRequest, CallOptions)

public virtual AsyncUnaryCall<FetchModelServerVersionsResponse> FetchModelServerVersionsAsync(FetchModelServerVersionsRequest request, CallOptions options)

Fetches available model server versions. Open-source servers use their own versioning schemas (e.g., vllm uses semver like v1.0.0).

Some model servers have different versioning schemas depending on the accelerator. For example, vllm uses semver on GPUs, but returns nightly build tags on TPUs. All available versions will be returned when different schemas are present.

Parameters
Name	Description
request	FetchModelServerVersionsRequest The request to send to the server.
options	CallOptions The options for the call.

Returns
Type	Description
AsyncUnaryCallFetchModelServerVersionsResponse	The call object.

FetchModelServerVersionsAsync(FetchModelServerVersionsRequest, Metadata, DateTime?, CancellationToken)

public virtual AsyncUnaryCall<FetchModelServerVersionsResponse> FetchModelServerVersionsAsync(FetchModelServerVersionsRequest request, Metadata headers = null, DateTime? deadline = null, CancellationToken cancellationToken = default)

Fetches available model server versions. Open-source servers use their own versioning schemas (e.g., vllm uses semver like v1.0.0).

Some model servers have different versioning schemas depending on the accelerator. For example, vllm uses semver on GPUs, but returns nightly build tags on TPUs. All available versions will be returned when different schemas are present.

Parameters
Name	Description
request	FetchModelServerVersionsRequest The request to send to the server.
headers	Metadata The initial metadata to send with the call. This parameter is optional.
deadline	System.DateTime An optional deadline for the call. The call will be cancelled if deadline is hit.
cancellationToken	CancellationToken An optional token for canceling the call.

Returns
Type	Description
AsyncUnaryCallFetchModelServerVersionsResponse	The call object.

FetchModelServers(FetchModelServersRequest, CallOptions)

public virtual FetchModelServersResponse FetchModelServers(FetchModelServersRequest request, CallOptions options)

Fetches available model servers. Open-source model servers use simplified, lowercase names (e.g., vllm).

Parameters
Name	Description
request	FetchModelServersRequest The request to send to the server.
options	CallOptions The options for the call.

Returns
Type	Description
FetchModelServersResponse	The response received from the server.

FetchModelServers(FetchModelServersRequest, Metadata, DateTime?, CancellationToken)

public virtual FetchModelServersResponse FetchModelServers(FetchModelServersRequest request, Metadata headers = null, DateTime? deadline = null, CancellationToken cancellationToken = default)

Fetches available model servers. Open-source model servers use simplified, lowercase names (e.g., vllm).

Parameters
Name	Description
request	FetchModelServersRequest The request to send to the server.
headers	Metadata The initial metadata to send with the call. This parameter is optional.
deadline	System.DateTime An optional deadline for the call. The call will be cancelled if deadline is hit.
cancellationToken	CancellationToken An optional token for canceling the call.

Returns
Type	Description
FetchModelServersResponse	The response received from the server.

FetchModelServersAsync(FetchModelServersRequest, CallOptions)

public virtual AsyncUnaryCall<FetchModelServersResponse> FetchModelServersAsync(FetchModelServersRequest request, CallOptions options)

Fetches available model servers. Open-source model servers use simplified, lowercase names (e.g., vllm).

Parameters
Name	Description
request	FetchModelServersRequest The request to send to the server.
options	CallOptions The options for the call.

Returns
Type	Description
AsyncUnaryCallFetchModelServersResponse	The call object.

FetchModelServersAsync(FetchModelServersRequest, Metadata, DateTime?, CancellationToken)

public virtual AsyncUnaryCall<FetchModelServersResponse> FetchModelServersAsync(FetchModelServersRequest request, Metadata headers = null, DateTime? deadline = null, CancellationToken cancellationToken = default)

Fetches available model servers. Open-source model servers use simplified, lowercase names (e.g., vllm).

Parameters
Name	Description
request	FetchModelServersRequest The request to send to the server.
headers	Metadata The initial metadata to send with the call. This parameter is optional.
deadline	System.DateTime An optional deadline for the call. The call will be cancelled if deadline is hit.
cancellationToken	CancellationToken An optional token for canceling the call.

Returns
Type	Description
AsyncUnaryCallFetchModelServersResponse	The call object.

FetchModels(FetchModelsRequest, CallOptions)

public virtual FetchModelsResponse FetchModels(FetchModelsRequest request, CallOptions options)

Fetches available models. Open-source models follow the Huggingface Hub owner/model_name format.

Parameters
Name	Description
request	FetchModelsRequest The request to send to the server.
options	CallOptions The options for the call.

Returns
Type	Description
FetchModelsResponse	The response received from the server.

FetchModels(FetchModelsRequest, Metadata, DateTime?, CancellationToken)

public virtual FetchModelsResponse FetchModels(FetchModelsRequest request, Metadata headers = null, DateTime? deadline = null, CancellationToken cancellationToken = default)

Fetches available models. Open-source models follow the Huggingface Hub owner/model_name format.

Parameters
Name	Description
request	FetchModelsRequest The request to send to the server.
headers	Metadata The initial metadata to send with the call. This parameter is optional.
deadline	System.DateTime An optional deadline for the call. The call will be cancelled if deadline is hit.
cancellationToken	CancellationToken An optional token for canceling the call.

Returns
Type	Description
FetchModelsResponse	The response received from the server.

FetchModelsAsync(FetchModelsRequest, CallOptions)

public virtual AsyncUnaryCall<FetchModelsResponse> FetchModelsAsync(FetchModelsRequest request, CallOptions options)

Fetches available models. Open-source models follow the Huggingface Hub owner/model_name format.

Parameters
Name	Description
request	FetchModelsRequest The request to send to the server.
options	CallOptions The options for the call.

Returns
Type	Description
AsyncUnaryCallFetchModelsResponse	The call object.

FetchModelsAsync(FetchModelsRequest, Metadata, DateTime?, CancellationToken)

public virtual AsyncUnaryCall<FetchModelsResponse> FetchModelsAsync(FetchModelsRequest request, Metadata headers = null, DateTime? deadline = null, CancellationToken cancellationToken = default)

Fetches available models. Open-source models follow the Huggingface Hub owner/model_name format.

Parameters
Name	Description
request	FetchModelsRequest The request to send to the server.
headers	Metadata The initial metadata to send with the call. This parameter is optional.
deadline	System.DateTime An optional deadline for the call. The call will be cancelled if deadline is hit.
cancellationToken	CancellationToken An optional token for canceling the call.

Returns
Type	Description
AsyncUnaryCallFetchModelsResponse	The call object.

FetchProfiles(FetchProfilesRequest, CallOptions)

public virtual FetchProfilesResponse FetchProfiles(FetchProfilesRequest request, CallOptions options)

Fetches available profiles. A profile contains performance metrics and cost information for a specific model server setup. Profiles can be filtered by parameters. If no filters are provided, all profiles are returned.

Profiles display a single value per performance metric based on the provided performance requirements. If no requirements are given, the metrics represent the inflection point. See Run best practice inference with GKE Inference Quickstart recipes for details.

Parameters
Name	Description
request	FetchProfilesRequest The request to send to the server.
options	CallOptions The options for the call.

Returns
Type	Description
FetchProfilesResponse	The response received from the server.

FetchProfiles(FetchProfilesRequest, Metadata, DateTime?, CancellationToken)

public virtual FetchProfilesResponse FetchProfiles(FetchProfilesRequest request, Metadata headers = null, DateTime? deadline = null, CancellationToken cancellationToken = default)

Fetches available profiles. A profile contains performance metrics and cost information for a specific model server setup. Profiles can be filtered by parameters. If no filters are provided, all profiles are returned.

Profiles display a single value per performance metric based on the provided performance requirements. If no requirements are given, the metrics represent the inflection point. See Run best practice inference with GKE Inference Quickstart recipes for details.

Parameters
Name	Description
request	FetchProfilesRequest The request to send to the server.
headers	Metadata The initial metadata to send with the call. This parameter is optional.
deadline	System.DateTime An optional deadline for the call. The call will be cancelled if deadline is hit.
cancellationToken	CancellationToken An optional token for canceling the call.

Returns
Type	Description
FetchProfilesResponse	The response received from the server.

FetchProfilesAsync(FetchProfilesRequest, CallOptions)

public virtual AsyncUnaryCall<FetchProfilesResponse> FetchProfilesAsync(FetchProfilesRequest request, CallOptions options)

Fetches available profiles. A profile contains performance metrics and cost information for a specific model server setup. Profiles can be filtered by parameters. If no filters are provided, all profiles are returned.

Profiles display a single value per performance metric based on the provided performance requirements. If no requirements are given, the metrics represent the inflection point. See Run best practice inference with GKE Inference Quickstart recipes for details.

Parameters
Name	Description
request	FetchProfilesRequest The request to send to the server.
options	CallOptions The options for the call.

Returns
Type	Description
AsyncUnaryCallFetchProfilesResponse	The call object.

FetchProfilesAsync(FetchProfilesRequest, Metadata, DateTime?, CancellationToken)

public virtual AsyncUnaryCall<FetchProfilesResponse> FetchProfilesAsync(FetchProfilesRequest request, Metadata headers = null, DateTime? deadline = null, CancellationToken cancellationToken = default)

Fetches available profiles. A profile contains performance metrics and cost information for a specific model server setup. Profiles can be filtered by parameters. If no filters are provided, all profiles are returned.

Profiles display a single value per performance metric based on the provided performance requirements. If no requirements are given, the metrics represent the inflection point. See Run best practice inference with GKE Inference Quickstart recipes for details.

Parameters
Name	Description
request	FetchProfilesRequest The request to send to the server.
headers	Metadata The initial metadata to send with the call. This parameter is optional.
deadline	System.DateTime An optional deadline for the call. The call will be cancelled if deadline is hit.
cancellationToken	CancellationToken An optional token for canceling the call.

Returns
Type	Description
AsyncUnaryCallFetchProfilesResponse	The call object.

GenerateOptimizedManifest(GenerateOptimizedManifestRequest, CallOptions)

public virtual GenerateOptimizedManifestResponse GenerateOptimizedManifest(GenerateOptimizedManifestRequest request, CallOptions options)

Generates an optimized deployment manifest for a given model and model server, based on the specified accelerator, performance targets, and configurations. See Run best practice inference with GKE Inference Quickstart recipes for deployment details.

Parameters
Name	Description
request	GenerateOptimizedManifestRequest The request to send to the server.
options	CallOptions The options for the call.

Returns
Type	Description
GenerateOptimizedManifestResponse	The response received from the server.

GenerateOptimizedManifest(GenerateOptimizedManifestRequest, Metadata, DateTime?, CancellationToken)

public virtual GenerateOptimizedManifestResponse GenerateOptimizedManifest(GenerateOptimizedManifestRequest request, Metadata headers = null, DateTime? deadline = null, CancellationToken cancellationToken = default)

Generates an optimized deployment manifest for a given model and model server, based on the specified accelerator, performance targets, and configurations. See Run best practice inference with GKE Inference Quickstart recipes for deployment details.

Parameters
Name	Description
request	GenerateOptimizedManifestRequest The request to send to the server.
headers	Metadata The initial metadata to send with the call. This parameter is optional.
deadline	System.DateTime An optional deadline for the call. The call will be cancelled if deadline is hit.
cancellationToken	CancellationToken An optional token for canceling the call.

Returns
Type	Description
GenerateOptimizedManifestResponse	The response received from the server.

GenerateOptimizedManifestAsync(GenerateOptimizedManifestRequest, CallOptions)

public virtual AsyncUnaryCall<GenerateOptimizedManifestResponse> GenerateOptimizedManifestAsync(GenerateOptimizedManifestRequest request, CallOptions options)

Generates an optimized deployment manifest for a given model and model server, based on the specified accelerator, performance targets, and configurations. See Run best practice inference with GKE Inference Quickstart recipes for deployment details.

Parameters
Name	Description
request	GenerateOptimizedManifestRequest The request to send to the server.
options	CallOptions The options for the call.

Returns
Type	Description
AsyncUnaryCallGenerateOptimizedManifestResponse	The call object.

GenerateOptimizedManifestAsync(GenerateOptimizedManifestRequest, Metadata, DateTime?, CancellationToken)

public virtual AsyncUnaryCall<GenerateOptimizedManifestResponse> GenerateOptimizedManifestAsync(GenerateOptimizedManifestRequest request, Metadata headers = null, DateTime? deadline = null, CancellationToken cancellationToken = default)

Generates an optimized deployment manifest for a given model and model server, based on the specified accelerator, performance targets, and configurations. See Run best practice inference with GKE Inference Quickstart recipes for deployment details.

Parameters
Name	Description
request	GenerateOptimizedManifestRequest The request to send to the server.
headers	Metadata The initial metadata to send with the call. This parameter is optional.
deadline	System.DateTime An optional deadline for the call. The call will be cancelled if deadline is hit.
cancellationToken	CancellationToken An optional token for canceling the call.

Returns
Type	Description
AsyncUnaryCallGenerateOptimizedManifestResponse	The call object.

NewInstance(ClientBaseConfiguration)

protected override GkeInferenceQuickstart.GkeInferenceQuickstartClient NewInstance(ClientBase.ClientBaseConfiguration configuration)

Creates a new instance of client from given ClientBaseConfiguration.

Parameter
Name	Description
configuration	ClientBaseClientBaseConfiguration

Returns
Type	Description
GkeInferenceQuickstartGkeInferenceQuickstartClient

Overrides

Grpc.Core.ClientBase<Google.Cloud.GkeRecommender.V1.GkeInferenceQuickstart.GkeInferenceQuickstartClient>.NewInstance(Grpc.Core.ClientBase.ClientBaseConfiguration)