SGLang

This document describes how to configure your Google Kubernetes Engine deployment so that you can use Google Cloud Managed Service for Prometheus to collect metrics from SGLang. This document shows you how to do the following:

These instructions apply only if you are using managed collection with Managed Service for Prometheus. If you are using self-deployed collection, then see the SGLang documentation for installation information.

These instructions are provided as an example and are expected to work in most Kubernetes environments. If you are having trouble installing an application or exporter due to restrictive security or organizational policies, then we recommend you consult open-source documentation for support.

For information about SGLang, see SGLang.

Prerequisites

To collect metrics from SGLang by using Managed Service for Prometheus and managed collection, your deployment must meet the following requirements:

  • Your cluster must be running Google Kubernetes Engine version 1.28.15-gke.2475000 or later.
  • You must be running Managed Service for Prometheus with managed collection enabled. For more information, see Get started with managed collection.

SGLang exposes Prometheus-format metrics when launched with the --enable-metrics flag; you do not have to install an exporter separately. To verify that SGLang is emitting metrics on the expected endpoints, do the following:

  1. Set up port forwarding by using the following command: 
    kubectl -n NAMESPACE_NAME port-forward POD_NAME 30000
  2. Access the localhost:30000/metrics endpoint by using the browser or the curl utility in another terminal session.

Use automatic application monitoring

SGLang supports the use of automatic application monitoring. When you use automatic application monitoring, Google Kubernetes Engine does the following:

  • Detects deployed instances of SGLang workloads.
  • Deploys a PodMonitoring resource for each detected workload instance.
  • Installs Cloud Monitoring dashboards for the SGLang metrics.

To use automatic application monitoring, you must enable the feature on your GKE cluster. You can use the Google Cloud console, the Google Cloud CLI (version 492.0.0 or later), or the GKE API. For more information, see Enable automatic application monitoring.

Define a PodMonitoring resource

For target discovery, the Managed Service for Prometheus Operator requires a PodMonitoring resource that corresponds to SGLang in the same namespace.

You can use the following PodMonitoring configuration:


apiVersion: monitoring.googleapis.com/v1
kind: PodMonitoring
metadata:
  name: sglang
  labels:
    app.kubernetes.io/name: sglang
    app.kubernetes.io/part-of: google-cloud-managed-prometheus
spec:
  endpoints:
  - port: 30000
    scheme: http
    interval: 15s
    path: /metrics
  selector:
    matchLabels:
      app: sglang-server
Ensure that the values of the port and matchLabels fields match those of the SGLang pods you want to monitor.

To apply configuration changes from a local file, run the following command: 

kubectl apply -n NAMESPACE_NAME -f FILE_NAME

You can also use Terraform to manage your configurations.

Verify the configuration

You can use Metrics Explorer to verify that you correctly configured SGLang. It might take one or two minutes for Cloud Monitoring to ingest your metrics.

To verify the metrics are ingested, do the following:

  1. In the Google Cloud console, go to the  Metrics explorer page:

    Go to Metrics explorer

    If you use the search bar to find this page, then select the result whose subheading is Monitoring.

  2. In the toolbar of the query-builder pane, select the button whose name is  PromQL.
  3. Enter and run the following query:
    up{job=~"sglang|gke-ac-.*", cluster="CLUSTER_NAME", namespace="NAMESPACE_NAME"}

Troubleshooting

For information about troubleshooting metric ingestion problems, see Problems with collection from exporters in Troubleshooting ingestion-side problems.