Google menggunakan teknologi AI untuk menerjemahkan konten ke dalam bahasa pilihan Anda. Terjemahan AI mungkin mengandung kesalahan.

Panduan pengguna model Gemma dengan Ollama di GDC dengan air gap

Dokumen ini memberikan petunjuk langkah demi langkah untuk men-deploy model open weight dari Google DeepMind, khususnya rangkaian Gemma 4, menggunakan Ollama di lingkungan Google Distributed Cloud (GDC) dengan air gap. Panduan ini membahas penggunaan Ollama untuk penyajian yang disederhanakan, menggunakan kemampuan platform GDC, termasuk resource Kubernetes, Harbor, dan GPU. Pilih model dan opsi GPU di bagian Model; setiap langkah lainnya identik untuk semua model.

Model yang didukung

Model	Tag Ollama	Parameter	Lisensi	Opsi GPU
Gemma 4 E2B	`gemma4:e2b`	5,1 M (2 M efektif)	Persyaratan Penggunaan Gemma	1 x GPU apa pun
Gemma 4 E4B	`gemma4:e4b`	8,0 M (4 M efektif)	Persyaratan Penggunaan Gemma	1 x GPU apa pun
Gemma 4 12B	`gemma4:12b`	12B (padat)	Persyaratan Penggunaan Gemma	1 x GPU apa pun
Gemma 4 26B-A4B	`gemma4:26b`	Total 25,8 miliar / 4 miliar aktif (mixture of experts)	Persyaratan Penggunaan Gemma	1 x GPU apa pun
Gemma 4 31B	`gemma4:31b`	31,3 miliar (dense)	Persyaratan Penggunaan Gemma	1 x GPU apa pun

Arsitektur

Solusi ini melibatkan deployment Ollama sebagai backend inferensi dalam container (Deployment) dalam cluster standar. Bobot model disimpan di Persistent Volume, diisi dari file yang didownload secara lokal dan diupload menggunakan pod loader. Service Kubernetes dengan jenis LoadBalancer mengekspos API backend, dan objek NetworkPolicy / ProjectNetworkPolicy mengamankan akses ke API tersebut.

Sebelum memulai

Selesaikan implementasi referensi Model Bobot Terbuka dengan Ollama di workstation, dan pastikan Anda memiliki akses ke direktori ${HOME}/gdcag-solutions.

Identity and Access Management

Panduan ini mengasumsikan bahwa Anda memiliki peran berikut di cluster standar:

Opsional: Standard Cluster Admin (Melihat cluster dari konsol)
Standard Cluster Admin (standard-cluster-admin)
StandardClusterRoleBinding ke StandardClusterRole cluster-admin (dibuat oleh Admin Project IAM, lihat bagian Identity and Access Management dari implementasi referensi)

Workstation

Jalankan semua perintah dalam panduan ini dari workstation dengan alat dan konektivitas yang diperlukan ke lingkungan dan internet, kecuali jika ada langkah yang menyatakan sebaliknya.

Konfigurasi lingkungan berlapis:

${HOME}/gdcag-solutions/env.sh memuat setelan platform dan registry.
${HOME}/gdcag-solutions/open-weight-models/ollama/env.sh menambahkan setelan Ollama.
Panduan pengguna ini menambahkan env.d/model.sh (model) dan env.d/accelerator.sh (opsi GPU dan setelan runtime), yang Anda tulis di bagian Model; dan env.d/model-common.sh (nilai yang berasal dari keduanya).

Buat struktur direktori panduan pengguna:

mkdir -p ${HOME}/gdcag-solutions/open-weight-models/ollama/gemma/env.d

Buat file konfigurasi lingkungan model umum. Fungsi ini menghitung nama dan jalur yang berasal dari nama model:

cat << 'EOF' > ${HOME}/gdcag-solutions/open-weight-models/ollama/gemma/env.d/model-common.sh && echo "Successfully created." || echo "Failed to create!"
# Model (derived from env.d/model.sh)
export GDCS_OLLAMA_MODEL_LOWERCASE="${GDCS_OLLAMA_MODEL,,}"
export GDCS_OLLAMA_MODEL_KUBERNETES="$(echo "${GDCS_OLLAMA_MODEL_LOWERCASE}" | tr ':' '-')"

# Paths
export GDCS_OLLAMA_MODEL_DOWNLOAD_DIR="${GDCS_USER_GUIDE_HOME}/models/${GDCS_OLLAMA_MODEL_LOWERCASE}"
export GDCS_OLLAMA_PV_MOUNT_PATH="/root/.ollama/models"

# Workload
export GDCS_WORKLOAD_NAMESPACE="ollama-gemma"
export GDCS_WORKLOAD_NAME="${GDCS_OLLAMA_MODEL_KUBERNETES}"
export GDCS_MODEL_LOADER_POD_NAME="loader-${GDCS_OLLAMA_MODEL_KUBERNETES}"
EOF

Buat file loader lingkungan panduan pengguna. File ini mengambil lingkungan root, lingkungan implementasi, dan lingkungan model:

cat << 'EOF' > ${HOME}/gdcag-solutions/open-weight-models/ollama/gemma/env.sh && echo "Successfully created." || echo "Failed to create!"
source "${HOME}/gdcag-solutions/open-weight-models/ollama/env.sh"

export GDCS_USER_GUIDE_HOME="${HOME}/gdcag-solutions/open-weight-models/ollama/gemma"
echo "GDCS_USER_GUIDE_HOME=${GDCS_USER_GUIDE_HOME}"

# Sourced in dependency order: the model selection, the accelerator and runtime settings, then the values derived from them
source "${GDCS_USER_GUIDE_HOME}/env.d/model.sh"
source "${GDCS_USER_GUIDE_HOME}/env.d/accelerator.sh"
source "${GDCS_USER_GUIDE_HOME}/env.d/model-common.sh"
EOF

Model

Setiap tab mencantumkan fakta model, perintah yang menulis file lingkungan model env.d/model.sh, dan tab bertingkat yang menulis file lingkungan akselerator env.d/accelerator.sh untuk opsi GPU yang Anda pilih dengan jenis resource yang telah diisi otomatis dan setelan Ollama. Untuk beralih model atau opsi GPU nanti, tulis file lingkungan yang berbeda, ambil sumber lingkungan lagi, dan ulangi bagian Implementasi; workload beberapa model dapat berjalan berdampingan di namespace yang sama karena setiap nama resource berasal dari nama model.

GPU	Memori	Jenis mesin	Jenis resource akselerator
NVIDIA A100	40 GB	`a2-highgpu-1g-gdc`	`nvidia.com/gpu-pod-NVIDIA_A100_PCIE_40GB`
NVIDIA A100	80 GB	`a2-ultragpu-1g-gdc`, `a2-ultragpu-2g-gdc`	`nvidia.com/gpu-pod-NVIDIA_A100_80GB_PCIE`
NVIDIA H100	94 GB	`a3-highgpu-1g-gdc` (Pratinjau), `a3-highgpu-2g-gdc` (Pratinjau), `a3-highgpu-4g-gdc`	`nvidia.com/gpu-pod-NVIDIA_H100_NVL`
NVIDIA H200	141 GB	`a3-ultragpu-8g-gdc`	`nvidia.com/gpu-pod-NVIDIA_H200`
NVIDIA B300	288 GB	`a4-ultragpu-8g-gdc`	`nvidia.com/gpu-pod-NVIDIA_B300_SXM6_AC`

Buat file lingkungan model dan akselerator untuk opsi model dan GPU yang ingin Anda deploy.

Gemma 4 E2B

Atribut	Nilai
Tag Ollama	`gemma4:e2b`
Parameter	5,1 M (2 M efektif)
Lisensi	Persyaratan Penggunaan Gemma
Bobot	Kuantisasi default Ollama, sekitar 4,6 GB
Panjang konteks	128 ribu token

Buat file lingkungan model:

cat << 'EOF' > ${HOME}/gdcag-solutions/open-weight-models/ollama/gemma/env.d/model.sh && echo "Successfully created." || echo "Failed to create!"
# Model: Gemma 4 E2B
export GDCS_OLLAMA_MODEL="gemma4:e2b"
EOF

Buat file lingkungan akselerator untuk opsi GPU Anda:

NVIDIA A100 40 GB

cat << 'EOF' > ${HOME}/gdcag-solutions/open-weight-models/ollama/gemma/env.d/accelerator.sh && echo "Successfully created." || echo "Failed to create!"
# Accelerator: NVIDIA A100 40 GB
export GDCS_WORKLOAD_ACCELERATOR_TYPE="nvidia.com/gpu-pod-NVIDIA_A100_PCIE_40GB"
export GDCS_WORKLOAD_ACCELERATOR_COUNT="1"

# Resources
export GDCS_WORKLOAD_CPU_LIMIT="4"
export GDCS_WORKLOAD_CPU_REQUEST="2"
export GDCS_WORKLOAD_MEMORY_LIMIT="12Gi"
export GDCS_WORKLOAD_MEMORY_REQUEST="6Gi"

# Ollama
export GDCS_OLLAMA_NUM_PARALLEL="4"
export GDCS_OLLAMA_TEST_NUM_PREDICT="128"
EOF

NVIDIA A100 80 GB

cat << 'EOF' > ${HOME}/gdcag-solutions/open-weight-models/ollama/gemma/env.d/accelerator.sh && echo "Successfully created." || echo "Failed to create!"
# Accelerator: NVIDIA A100 80 GB
export GDCS_WORKLOAD_ACCELERATOR_TYPE="nvidia.com/gpu-pod-NVIDIA_A100_80GB_PCIE"
export GDCS_WORKLOAD_ACCELERATOR_COUNT="1"

# Resources
export GDCS_WORKLOAD_CPU_LIMIT="4"
export GDCS_WORKLOAD_CPU_REQUEST="2"
export GDCS_WORKLOAD_MEMORY_LIMIT="12Gi"
export GDCS_WORKLOAD_MEMORY_REQUEST="6Gi"

# Ollama
export GDCS_OLLAMA_NUM_PARALLEL="4"
export GDCS_OLLAMA_TEST_NUM_PREDICT="128"
EOF

NVIDIA H100 94 GB

cat << 'EOF' > ${HOME}/gdcag-solutions/open-weight-models/ollama/gemma/env.d/accelerator.sh && echo "Successfully created." || echo "Failed to create!"
# Accelerator: NVIDIA H100 94 GB
export GDCS_WORKLOAD_ACCELERATOR_TYPE="nvidia.com/gpu-pod-NVIDIA_H100_NVL"
export GDCS_WORKLOAD_ACCELERATOR_COUNT="1"

# Resources
export GDCS_WORKLOAD_CPU_LIMIT="4"
export GDCS_WORKLOAD_CPU_REQUEST="2"
export GDCS_WORKLOAD_MEMORY_LIMIT="12Gi"
export GDCS_WORKLOAD_MEMORY_REQUEST="6Gi"

# Ollama
export GDCS_OLLAMA_NUM_PARALLEL="4"
export GDCS_OLLAMA_TEST_NUM_PREDICT="128"
EOF

NVIDIA H200 141 GB

cat << 'EOF' > ${HOME}/gdcag-solutions/open-weight-models/ollama/gemma/env.d/accelerator.sh && echo "Successfully created." || echo "Failed to create!"
# Accelerator: NVIDIA H200 141 GB
export GDCS_WORKLOAD_ACCELERATOR_TYPE="nvidia.com/gpu-pod-NVIDIA_H200"
export GDCS_WORKLOAD_ACCELERATOR_COUNT="1"

# Resources
export GDCS_WORKLOAD_CPU_LIMIT="4"
export GDCS_WORKLOAD_CPU_REQUEST="2"
export GDCS_WORKLOAD_MEMORY_LIMIT="12Gi"
export GDCS_WORKLOAD_MEMORY_REQUEST="6Gi"

# Ollama
export GDCS_OLLAMA_NUM_PARALLEL="4"
export GDCS_OLLAMA_TEST_NUM_PREDICT="128"
EOF

NVIDIA B300 288 GB

cat << 'EOF' > ${HOME}/gdcag-solutions/open-weight-models/ollama/gemma/env.d/accelerator.sh && echo "Successfully created." || echo "Failed to create!"
# Accelerator: NVIDIA B300 288 GB
export GDCS_WORKLOAD_ACCELERATOR_TYPE="nvidia.com/gpu-pod-NVIDIA_B300_SXM6_AC"
export GDCS_WORKLOAD_ACCELERATOR_COUNT="1"

# Resources
export GDCS_WORKLOAD_CPU_LIMIT="4"
export GDCS_WORKLOAD_CPU_REQUEST="2"
export GDCS_WORKLOAD_MEMORY_LIMIT="12Gi"
export GDCS_WORKLOAD_MEMORY_REQUEST="6Gi"

# Ollama
export GDCS_OLLAMA_NUM_PARALLEL="4"
export GDCS_OLLAMA_TEST_NUM_PREDICT="128"
EOF

Gemma 4 E4B

Atribut	Nilai
Tag Ollama	`gemma4:e4b`
Parameter	8,0 M (4 M efektif)
Lisensi	Persyaratan Penggunaan Gemma
Bobot	Kuantisasi default Ollama, sekitar 6,6 GB
Panjang konteks	128 ribu token

Buat file lingkungan model:

cat << 'EOF' > ${HOME}/gdcag-solutions/open-weight-models/ollama/gemma/env.d/model.sh && echo "Successfully created." || echo "Failed to create!"
# Model: Gemma 4 E4B
export GDCS_OLLAMA_MODEL="gemma4:e4b"
EOF

Buat file lingkungan akselerator untuk opsi GPU Anda:

NVIDIA A100 40 GB

cat << 'EOF' > ${HOME}/gdcag-solutions/open-weight-models/ollama/gemma/env.d/accelerator.sh && echo "Successfully created." || echo "Failed to create!"
# Accelerator: NVIDIA A100 40 GB
export GDCS_WORKLOAD_ACCELERATOR_TYPE="nvidia.com/gpu-pod-NVIDIA_A100_PCIE_40GB"
export GDCS_WORKLOAD_ACCELERATOR_COUNT="1"

# Resources
export GDCS_WORKLOAD_CPU_LIMIT="4"
export GDCS_WORKLOAD_CPU_REQUEST="2"
export GDCS_WORKLOAD_MEMORY_LIMIT="16Gi"
export GDCS_WORKLOAD_MEMORY_REQUEST="8Gi"

# Ollama
export GDCS_OLLAMA_NUM_PARALLEL="4"
export GDCS_OLLAMA_TEST_NUM_PREDICT="128"
EOF

NVIDIA A100 80 GB

cat << 'EOF' > ${HOME}/gdcag-solutions/open-weight-models/ollama/gemma/env.d/accelerator.sh && echo "Successfully created." || echo "Failed to create!"
# Accelerator: NVIDIA A100 80 GB
export GDCS_WORKLOAD_ACCELERATOR_TYPE="nvidia.com/gpu-pod-NVIDIA_A100_80GB_PCIE"
export GDCS_WORKLOAD_ACCELERATOR_COUNT="1"

# Resources
export GDCS_WORKLOAD_CPU_LIMIT="4"
export GDCS_WORKLOAD_CPU_REQUEST="2"
export GDCS_WORKLOAD_MEMORY_LIMIT="16Gi"
export GDCS_WORKLOAD_MEMORY_REQUEST="8Gi"

# Ollama
export GDCS_OLLAMA_NUM_PARALLEL="4"
export GDCS_OLLAMA_TEST_NUM_PREDICT="128"
EOF

NVIDIA H100 94 GB

cat << 'EOF' > ${HOME}/gdcag-solutions/open-weight-models/ollama/gemma/env.d/accelerator.sh && echo "Successfully created." || echo "Failed to create!"
# Accelerator: NVIDIA H100 94 GB
export GDCS_WORKLOAD_ACCELERATOR_TYPE="nvidia.com/gpu-pod-NVIDIA_H100_NVL"
export GDCS_WORKLOAD_ACCELERATOR_COUNT="1"

# Resources
export GDCS_WORKLOAD_CPU_LIMIT="4"
export GDCS_WORKLOAD_CPU_REQUEST="2"
export GDCS_WORKLOAD_MEMORY_LIMIT="16Gi"
export GDCS_WORKLOAD_MEMORY_REQUEST="8Gi"

# Ollama
export GDCS_OLLAMA_NUM_PARALLEL="4"
export GDCS_OLLAMA_TEST_NUM_PREDICT="128"
EOF

NVIDIA H200 141 GB

cat << 'EOF' > ${HOME}/gdcag-solutions/open-weight-models/ollama/gemma/env.d/accelerator.sh && echo "Successfully created." || echo "Failed to create!"
# Accelerator: NVIDIA H200 141 GB
export GDCS_WORKLOAD_ACCELERATOR_TYPE="nvidia.com/gpu-pod-NVIDIA_H200"
export GDCS_WORKLOAD_ACCELERATOR_COUNT="1"

# Resources
export GDCS_WORKLOAD_CPU_LIMIT="4"
export GDCS_WORKLOAD_CPU_REQUEST="2"
export GDCS_WORKLOAD_MEMORY_LIMIT="16Gi"
export GDCS_WORKLOAD_MEMORY_REQUEST="8Gi"

# Ollama
export GDCS_OLLAMA_NUM_PARALLEL="4"
export GDCS_OLLAMA_TEST_NUM_PREDICT="128"
EOF

NVIDIA B300 288 GB

cat << 'EOF' > ${HOME}/gdcag-solutions/open-weight-models/ollama/gemma/env.d/accelerator.sh && echo "Successfully created." || echo "Failed to create!"
# Accelerator: NVIDIA B300 288 GB
export GDCS_WORKLOAD_ACCELERATOR_TYPE="nvidia.com/gpu-pod-NVIDIA_B300_SXM6_AC"
export GDCS_WORKLOAD_ACCELERATOR_COUNT="1"

# Resources
export GDCS_WORKLOAD_CPU_LIMIT="4"
export GDCS_WORKLOAD_CPU_REQUEST="2"
export GDCS_WORKLOAD_MEMORY_LIMIT="16Gi"
export GDCS_WORKLOAD_MEMORY_REQUEST="8Gi"

# Ollama
export GDCS_OLLAMA_NUM_PARALLEL="4"
export GDCS_OLLAMA_TEST_NUM_PREDICT="128"
EOF

Gemma 4 12B

Atribut	Nilai
Tag Ollama	`gemma4:12b`
Parameter	12B (padat)
Lisensi	Persyaratan Penggunaan Gemma
Bobot	Kuantisasi default Ollama, sekitar 7,7 GB
Panjang konteks	256 ribu token

Buat file lingkungan model:

cat << 'EOF' > ${HOME}/gdcag-solutions/open-weight-models/ollama/gemma/env.d/model.sh && echo "Successfully created." || echo "Failed to create!"
# Model: Gemma 4 12B
export GDCS_OLLAMA_MODEL="gemma4:12b"
EOF

Buat file lingkungan akselerator untuk opsi GPU Anda:

NVIDIA A100 40 GB

cat << 'EOF' > ${HOME}/gdcag-solutions/open-weight-models/ollama/gemma/env.d/accelerator.sh && echo "Successfully created." || echo "Failed to create!"
# Accelerator: NVIDIA A100 40 GB
export GDCS_WORKLOAD_ACCELERATOR_TYPE="nvidia.com/gpu-pod-NVIDIA_A100_PCIE_40GB"
export GDCS_WORKLOAD_ACCELERATOR_COUNT="1"

# Resources
export GDCS_WORKLOAD_CPU_LIMIT="4"
export GDCS_WORKLOAD_CPU_REQUEST="2"
export GDCS_WORKLOAD_MEMORY_LIMIT="16Gi"
export GDCS_WORKLOAD_MEMORY_REQUEST="8Gi"

# Ollama
export GDCS_OLLAMA_NUM_PARALLEL="4"
export GDCS_OLLAMA_TEST_NUM_PREDICT="128"
EOF

NVIDIA A100 80 GB

cat << 'EOF' > ${HOME}/gdcag-solutions/open-weight-models/ollama/gemma/env.d/accelerator.sh && echo "Successfully created." || echo "Failed to create!"
# Accelerator: NVIDIA A100 80 GB
export GDCS_WORKLOAD_ACCELERATOR_TYPE="nvidia.com/gpu-pod-NVIDIA_A100_80GB_PCIE"
export GDCS_WORKLOAD_ACCELERATOR_COUNT="1"

# Resources
export GDCS_WORKLOAD_CPU_LIMIT="4"
export GDCS_WORKLOAD_CPU_REQUEST="2"
export GDCS_WORKLOAD_MEMORY_LIMIT="16Gi"
export GDCS_WORKLOAD_MEMORY_REQUEST="8Gi"

# Ollama
export GDCS_OLLAMA_NUM_PARALLEL="4"
export GDCS_OLLAMA_TEST_NUM_PREDICT="128"
EOF

NVIDIA H100 94 GB

cat << 'EOF' > ${HOME}/gdcag-solutions/open-weight-models/ollama/gemma/env.d/accelerator.sh && echo "Successfully created." || echo "Failed to create!"
# Accelerator: NVIDIA H100 94 GB
export GDCS_WORKLOAD_ACCELERATOR_TYPE="nvidia.com/gpu-pod-NVIDIA_H100_NVL"
export GDCS_WORKLOAD_ACCELERATOR_COUNT="1"

# Resources
export GDCS_WORKLOAD_CPU_LIMIT="4"
export GDCS_WORKLOAD_CPU_REQUEST="2"
export GDCS_WORKLOAD_MEMORY_LIMIT="16Gi"
export GDCS_WORKLOAD_MEMORY_REQUEST="8Gi"

# Ollama
export GDCS_OLLAMA_NUM_PARALLEL="4"
export GDCS_OLLAMA_TEST_NUM_PREDICT="128"
EOF

NVIDIA H200 141 GB

cat << 'EOF' > ${HOME}/gdcag-solutions/open-weight-models/ollama/gemma/env.d/accelerator.sh && echo "Successfully created." || echo "Failed to create!"
# Accelerator: NVIDIA H200 141 GB
export GDCS_WORKLOAD_ACCELERATOR_TYPE="nvidia.com/gpu-pod-NVIDIA_H200"
export GDCS_WORKLOAD_ACCELERATOR_COUNT="1"

# Resources
export GDCS_WORKLOAD_CPU_LIMIT="4"
export GDCS_WORKLOAD_CPU_REQUEST="2"
export GDCS_WORKLOAD_MEMORY_LIMIT="16Gi"
export GDCS_WORKLOAD_MEMORY_REQUEST="8Gi"

# Ollama
export GDCS_OLLAMA_NUM_PARALLEL="4"
export GDCS_OLLAMA_TEST_NUM_PREDICT="128"
EOF

NVIDIA B300 288 GB

cat << 'EOF' > ${HOME}/gdcag-solutions/open-weight-models/ollama/gemma/env.d/accelerator.sh && echo "Successfully created." || echo "Failed to create!"
# Accelerator: NVIDIA B300 288 GB
export GDCS_WORKLOAD_ACCELERATOR_TYPE="nvidia.com/gpu-pod-NVIDIA_B300_SXM6_AC"
export GDCS_WORKLOAD_ACCELERATOR_COUNT="1"

# Resources
export GDCS_WORKLOAD_CPU_LIMIT="4"
export GDCS_WORKLOAD_CPU_REQUEST="2"
export GDCS_WORKLOAD_MEMORY_LIMIT="16Gi"
export GDCS_WORKLOAD_MEMORY_REQUEST="8Gi"

# Ollama
export GDCS_OLLAMA_NUM_PARALLEL="4"
export GDCS_OLLAMA_TEST_NUM_PREDICT="128"
EOF

Gemma 4 26B-A4B

Atribut	Nilai
Tag Ollama	`gemma4:26b`
Parameter	Total 25,8 miliar / 4 miliar aktif (mixture of experts)
Lisensi	Persyaratan Penggunaan Gemma
Bobot	Kuantisasi default Ollama, sekitar 16 GB
Panjang konteks	256 ribu token

Buat file lingkungan model:

cat << 'EOF' > ${HOME}/gdcag-solutions/open-weight-models/ollama/gemma/env.d/model.sh && echo "Successfully created." || echo "Failed to create!"
# Model: Gemma 4 26B-A4B
export GDCS_OLLAMA_MODEL="gemma4:26b"
EOF

Buat file lingkungan akselerator untuk opsi GPU Anda:

NVIDIA A100 40 GB

cat << 'EOF' > ${HOME}/gdcag-solutions/open-weight-models/ollama/gemma/env.d/accelerator.sh && echo "Successfully created." || echo "Failed to create!"
# Accelerator: NVIDIA A100 40 GB
export GDCS_WORKLOAD_ACCELERATOR_TYPE="nvidia.com/gpu-pod-NVIDIA_A100_PCIE_40GB"
export GDCS_WORKLOAD_ACCELERATOR_COUNT="1"

# Resources
export GDCS_WORKLOAD_CPU_LIMIT="4"
export GDCS_WORKLOAD_CPU_REQUEST="2"
export GDCS_WORKLOAD_MEMORY_LIMIT="32Gi"
export GDCS_WORKLOAD_MEMORY_REQUEST="16Gi"

# Ollama
export GDCS_OLLAMA_NUM_PARALLEL="4"
export GDCS_OLLAMA_TEST_NUM_PREDICT="128"
EOF

NVIDIA A100 80 GB

cat << 'EOF' > ${HOME}/gdcag-solutions/open-weight-models/ollama/gemma/env.d/accelerator.sh && echo "Successfully created." || echo "Failed to create!"
# Accelerator: NVIDIA A100 80 GB
export GDCS_WORKLOAD_ACCELERATOR_TYPE="nvidia.com/gpu-pod-NVIDIA_A100_80GB_PCIE"
export GDCS_WORKLOAD_ACCELERATOR_COUNT="1"

# Resources
export GDCS_WORKLOAD_CPU_LIMIT="4"
export GDCS_WORKLOAD_CPU_REQUEST="2"
export GDCS_WORKLOAD_MEMORY_LIMIT="32Gi"
export GDCS_WORKLOAD_MEMORY_REQUEST="16Gi"

# Ollama
export GDCS_OLLAMA_NUM_PARALLEL="4"
export GDCS_OLLAMA_TEST_NUM_PREDICT="128"
EOF

NVIDIA H100 94 GB

cat << 'EOF' > ${HOME}/gdcag-solutions/open-weight-models/ollama/gemma/env.d/accelerator.sh && echo "Successfully created." || echo "Failed to create!"
# Accelerator: NVIDIA H100 94 GB
export GDCS_WORKLOAD_ACCELERATOR_TYPE="nvidia.com/gpu-pod-NVIDIA_H100_NVL"
export GDCS_WORKLOAD_ACCELERATOR_COUNT="1"

# Resources
export GDCS_WORKLOAD_CPU_LIMIT="4"
export GDCS_WORKLOAD_CPU_REQUEST="2"
export GDCS_WORKLOAD_MEMORY_LIMIT="32Gi"
export GDCS_WORKLOAD_MEMORY_REQUEST="16Gi"

# Ollama
export GDCS_OLLAMA_NUM_PARALLEL="4"
export GDCS_OLLAMA_TEST_NUM_PREDICT="128"
EOF

NVIDIA H200 141 GB

cat << 'EOF' > ${HOME}/gdcag-solutions/open-weight-models/ollama/gemma/env.d/accelerator.sh && echo "Successfully created." || echo "Failed to create!"
# Accelerator: NVIDIA H200 141 GB
export GDCS_WORKLOAD_ACCELERATOR_TYPE="nvidia.com/gpu-pod-NVIDIA_H200"
export GDCS_WORKLOAD_ACCELERATOR_COUNT="1"

# Resources
export GDCS_WORKLOAD_CPU_LIMIT="4"
export GDCS_WORKLOAD_CPU_REQUEST="2"
export GDCS_WORKLOAD_MEMORY_LIMIT="32Gi"
export GDCS_WORKLOAD_MEMORY_REQUEST="16Gi"

# Ollama
export GDCS_OLLAMA_NUM_PARALLEL="4"
export GDCS_OLLAMA_TEST_NUM_PREDICT="128"
EOF

NVIDIA B300 288 GB

cat << 'EOF' > ${HOME}/gdcag-solutions/open-weight-models/ollama/gemma/env.d/accelerator.sh && echo "Successfully created." || echo "Failed to create!"
# Accelerator: NVIDIA B300 288 GB
export GDCS_WORKLOAD_ACCELERATOR_TYPE="nvidia.com/gpu-pod-NVIDIA_B300_SXM6_AC"
export GDCS_WORKLOAD_ACCELERATOR_COUNT="1"

# Resources
export GDCS_WORKLOAD_CPU_LIMIT="4"
export GDCS_WORKLOAD_CPU_REQUEST="2"
export GDCS_WORKLOAD_MEMORY_LIMIT="32Gi"
export GDCS_WORKLOAD_MEMORY_REQUEST="16Gi"

# Ollama
export GDCS_OLLAMA_NUM_PARALLEL="4"
export GDCS_OLLAMA_TEST_NUM_PREDICT="128"
EOF

Gemma 4 31B

Atribut	Nilai
Tag Ollama	`gemma4:31b`
Parameter	31,3 miliar (dense)
Lisensi	Persyaratan Penggunaan Gemma
Bobot	Kuantisasi default Ollama, sekitar 19 GB
Panjang konteks	256 ribu token

Buat file lingkungan model:

cat << 'EOF' > ${HOME}/gdcag-solutions/open-weight-models/ollama/gemma/env.d/model.sh && echo "Successfully created." || echo "Failed to create!"
# Model: Gemma 4 31B
export GDCS_OLLAMA_MODEL="gemma4:31b"
EOF

Buat file lingkungan akselerator untuk opsi GPU Anda:

NVIDIA A100 40 GB

cat << 'EOF' > ${HOME}/gdcag-solutions/open-weight-models/ollama/gemma/env.d/accelerator.sh && echo "Successfully created." || echo "Failed to create!"
# Accelerator: NVIDIA A100 40 GB
export GDCS_WORKLOAD_ACCELERATOR_TYPE="nvidia.com/gpu-pod-NVIDIA_A100_PCIE_40GB"
export GDCS_WORKLOAD_ACCELERATOR_COUNT="1"

# Resources
export GDCS_WORKLOAD_CPU_LIMIT="4"
export GDCS_WORKLOAD_CPU_REQUEST="2"
export GDCS_WORKLOAD_MEMORY_LIMIT="40Gi"
export GDCS_WORKLOAD_MEMORY_REQUEST="20Gi"

# Ollama
export GDCS_OLLAMA_NUM_PARALLEL="4"
export GDCS_OLLAMA_TEST_NUM_PREDICT="128"
EOF

NVIDIA A100 80 GB

cat << 'EOF' > ${HOME}/gdcag-solutions/open-weight-models/ollama/gemma/env.d/accelerator.sh && echo "Successfully created." || echo "Failed to create!"
# Accelerator: NVIDIA A100 80 GB
export GDCS_WORKLOAD_ACCELERATOR_TYPE="nvidia.com/gpu-pod-NVIDIA_A100_80GB_PCIE"
export GDCS_WORKLOAD_ACCELERATOR_COUNT="1"

# Resources
export GDCS_WORKLOAD_CPU_LIMIT="4"
export GDCS_WORKLOAD_CPU_REQUEST="2"
export GDCS_WORKLOAD_MEMORY_LIMIT="40Gi"
export GDCS_WORKLOAD_MEMORY_REQUEST="20Gi"

# Ollama
export GDCS_OLLAMA_NUM_PARALLEL="4"
export GDCS_OLLAMA_TEST_NUM_PREDICT="128"
EOF

NVIDIA H100 94 GB

cat << 'EOF' > ${HOME}/gdcag-solutions/open-weight-models/ollama/gemma/env.d/accelerator.sh && echo "Successfully created." || echo "Failed to create!"
# Accelerator: NVIDIA H100 94 GB
export GDCS_WORKLOAD_ACCELERATOR_TYPE="nvidia.com/gpu-pod-NVIDIA_H100_NVL"
export GDCS_WORKLOAD_ACCELERATOR_COUNT="1"

# Resources
export GDCS_WORKLOAD_CPU_LIMIT="4"
export GDCS_WORKLOAD_CPU_REQUEST="2"
export GDCS_WORKLOAD_MEMORY_LIMIT="40Gi"
export GDCS_WORKLOAD_MEMORY_REQUEST="20Gi"

# Ollama
export GDCS_OLLAMA_NUM_PARALLEL="4"
export GDCS_OLLAMA_TEST_NUM_PREDICT="128"
EOF

NVIDIA H200 141 GB

cat << 'EOF' > ${HOME}/gdcag-solutions/open-weight-models/ollama/gemma/env.d/accelerator.sh && echo "Successfully created." || echo "Failed to create!"
# Accelerator: NVIDIA H200 141 GB
export GDCS_WORKLOAD_ACCELERATOR_TYPE="nvidia.com/gpu-pod-NVIDIA_H200"
export GDCS_WORKLOAD_ACCELERATOR_COUNT="1"

# Resources
export GDCS_WORKLOAD_CPU_LIMIT="4"
export GDCS_WORKLOAD_CPU_REQUEST="2"
export GDCS_WORKLOAD_MEMORY_LIMIT="40Gi"
export GDCS_WORKLOAD_MEMORY_REQUEST="20Gi"

# Ollama
export GDCS_OLLAMA_NUM_PARALLEL="4"
export GDCS_OLLAMA_TEST_NUM_PREDICT="128"
EOF

NVIDIA B300 288 GB

cat << 'EOF' > ${HOME}/gdcag-solutions/open-weight-models/ollama/gemma/env.d/accelerator.sh && echo "Successfully created." || echo "Failed to create!"
# Accelerator: NVIDIA B300 288 GB
export GDCS_WORKLOAD_ACCELERATOR_TYPE="nvidia.com/gpu-pod-NVIDIA_B300_SXM6_AC"
export GDCS_WORKLOAD_ACCELERATOR_COUNT="1"

# Resources
export GDCS_WORKLOAD_CPU_LIMIT="4"
export GDCS_WORKLOAD_CPU_REQUEST="2"
export GDCS_WORKLOAD_MEMORY_LIMIT="40Gi"
export GDCS_WORKLOAD_MEMORY_REQUEST="20Gi"

# Ollama
export GDCS_OLLAMA_NUM_PARALLEL="4"
export GDCS_OLLAMA_TEST_NUM_PREDICT="128"
EOF

Ambil file lingkungan:

source ${HOME}/gdcag-solutions/open-weight-models/ollama/gemma/env.sh

Cluster

Ambil kredensial cluster:

gdcloud clusters get-credentials "${GDC_STANDARD_CLUSTER_NAME}" \
--project="${GDC_PROJECT}"

Pastikan jenis resource akselerator dari file lingkungan akselerator tersedia di cluster:
```
kubectl get nodes \
-o custom-columns='NAME:.metadata.name,ACCELERATOR:.status.allocatable.'"${GDCS_WORKLOAD_ACCELERATOR_TYPE}"
```
Catatan: Jika jumlah resource adalah 0 atau <none>, berarti cluster tidak memiliki node dengan jenis GPU ini atau node pool masih dalam proses pembuatan. Periksa jenis yang tersedia dengan kubectl get nodes -L node.cluster.private.gdc.goog/machine-class. Untuk menggunakan GPU lain, tetapkan GDCS_WORKLOAD_ACCELERATOR_TYPE di env.d/accelerator.sh ke salah satu jenis resource yang tercantum, sumberkan lingkungan lagi, dan jalankan kembali langkah ini.

Penerapan

Namespace

Buat namespace:

kubectl create namespace "${GDCS_WORKLOAD_NAMESPACE}" \
--dry-run=client \
--output=yaml | kubectl apply -f -

Tambahkan imagePullSecret:

kubectl create secret docker-registry "${GDCS_HARBOR_K8S_PULL_SECRET}" \
--dry-run=client \
--from-file=.dockerconfigjson=${GDCS_HARBOR_K8S_DOCKER_CONFIG}/config.json \
--namespace="${GDCS_WORKLOAD_NAMESPACE}" \
--output=yaml | kubectl apply -f -

Bobot model

Download bobot model:

${GDCS_IMPLEMENTATION_HOME}/ollama_model_weight_download.sh

Hitung ukuran PersistentVolumeClaim yang diperlukan dari bobot model yang didownload dengan overhead 25% (dibulatkan ke atas ke GiB utuh berikutnya):

MODEL_KIB=$(du -sk "${GDCS_OLLAMA_MODEL_DOWNLOAD_DIR}" | awk '{print $1}')
export GDCS_MODEL_PVC_SIZE="$(( (MODEL_KIB * 125 / 100 + 1048575) / 1048576 ))Gi"
echo "GDCS_MODEL_PVC_SIZE=${GDCS_MODEL_PVC_SIZE}"

Buat manifes untuk PersistentVolumeClaim untuk bobot model:

cat <<EOF > "${GDCS_USER_GUIDE_HOME}/${GDCS_WORKLOAD_NAME}-pvc.yaml" && echo "Successfully created." || echo "Failed to create!"
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: ${GDCS_WORKLOAD_NAME}-rwo
  namespace: ${GDCS_WORKLOAD_NAMESPACE}
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: ${GDCS_MODEL_PVC_SIZE}
EOF

Terapkan manifes untuk PersistentVolumeClaim untuk bobot model:

kubectl apply --filename="${GDCS_USER_GUIDE_HOME}/${GDCS_WORKLOAD_NAME}-pvc.yaml"

Tunggu hingga PersistentVolumeClaim mengikat dan mencetak nilai yang diperlukan Operator Infrastruktur untuk menyesuaikan kebijakan kualitas layanan (QoS) I/O:

kubectl wait \
--for=jsonpath='{.status.phase}'=Bound \
"pvc/${GDCS_WORKLOAD_NAME}-rwo" \
--namespace="${GDCS_WORKLOAD_NAMESPACE}" \
--timeout=120s
BACKING_PVC=$(kubectl get pvc "${GDCS_WORKLOAD_NAME}-rwo" \
--namespace="${GDCS_WORKLOAD_NAMESPACE}" \
--output=jsonpath='{.spec.volumeName}')
echo "BACKING_PVC=${BACKING_PVC} ORG=${GDC_ORG}"

Jika performance-rwo tidak tersedia di cluster Anda, minta Operator Infrastruktur Anda untuk menerapkan kebijakan QoS berperforma tinggi (system-performance-<ORG>-admin, 30 IOPS per GiB dengan batas minimum 6.148 IOPS) ke volume yang mendukung PersistentVolumeClaim. Berikan nilai BACKING_PVC dan ORG yang dicetak pada langkah sebelumnya. Operator Infrastruktur menerapkan kebijakan seperti yang dijelaskan dalam runbook FILE-R0097 dalam dokumentasi Operator Infrastruktur; kebijakan ini berlaku pada volume aktif, sehingga Anda dapat melanjutkan ke langkah berikutnya.

Catatan: Jika block storage berperforma tinggi diaktifkan di cluster, Anda dapat menetapkan storageClassName: performance-rwo di manifes PersistentVolumeClaim, bukan meminta kebijakan QoS. Jika kubectl get storageclass performance-rwo tidak menampilkan apa pun, Operator Infrastruktur belum mengaktifkan block storage berperforma tinggi.

Buat manifes untuk loader model Pod:

cat <<EOF > "${GDCS_USER_GUIDE_HOME}/${GDCS_WORKLOAD_NAME}-loader.yaml" && echo "Successfully created." || echo "Failed to create!"
apiVersion: v1
kind: Pod
metadata:
  name: ${GDCS_MODEL_LOADER_POD_NAME}
  namespace: ${GDCS_WORKLOAD_NAMESPACE}
spec:
  containers:
    - command: ["/bin/sh", "-c", "trap 'exit 0' TERM; while :; do sleep 1; done"]
      image: ${GDCS_HARBOR_PROJECT_URI}/busybox/busybox:${GDCS_BUSYBOX_IMAGE_TAG}
      imagePullPolicy: IfNotPresent
      name: loader
      volumeMounts:
        - mountPath: ${GDCS_OLLAMA_PV_MOUNT_PATH}
          name: model-cache
  imagePullSecrets:
    - name: ${GDCS_HARBOR_K8S_PULL_SECRET}
  restartPolicy: Never
  volumes:
    - name: model-cache
      persistentVolumeClaim:
        claimName: ${GDCS_WORKLOAD_NAME}-rwo
EOF

Terapkan manifes untuk pemuat model Pod:

kubectl apply --filename="${GDCS_USER_GUIDE_HOME}/${GDCS_WORKLOAD_NAME}-loader.yaml"

Tunggu hingga pemuat model Pod berstatus Running:

kubectl wait \
--for=jsonpath='{.status.phase}'=Running \
"pod/${GDCS_MODEL_LOADER_POD_NAME}" \
--namespace="${GDCS_WORKLOAD_NAMESPACE}" \
--timeout=120s

Upload bobot model:

${GDCS_IMPLEMENTATION_HOME}/model_weight_upload.sh \
--local-dir="${GDCS_OLLAMA_MODEL_DOWNLOAD_DIR}" \
--pod="${GDCS_MODEL_LOADER_POD_NAME}" \
--target-dir="${GDCS_OLLAMA_PV_MOUNT_PATH}"

Hapus loader model Pod:

kubectl delete pod "${GDCS_MODEL_LOADER_POD_NAME}" \
--namespace="${GDCS_WORKLOAD_NAMESPACE}"

Server inferensi

Buat manifes untuk Deployment Ollama:

cat <<EOF > "${GDCS_USER_GUIDE_HOME}/${GDCS_WORKLOAD_NAME}-deployment.yaml" && echo "Successfully created." || echo "Failed to create!"
apiVersion: v1
kind: ConfigMap
metadata:
  name: ${GDCS_WORKLOAD_NAME}
data:
  OLLAMA_FLASH_ATTENTION: "1"
  OLLAMA_HOST: "0.0.0.0"
  OLLAMA_KEEP_ALIVE: "-1"
  OLLAMA_MODELS: "${GDCS_OLLAMA_PV_MOUNT_PATH}"
  OLLAMA_NUM_PARALLEL: "${GDCS_OLLAMA_NUM_PARALLEL}"
  OLLAMA_ORIGINS: "*"
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: ${GDCS_WORKLOAD_NAME}
  labels:
    app: ${GDCS_WORKLOAD_NAME}
spec:
  replicas: 1
  selector:
    matchLabels:
      app: ${GDCS_WORKLOAD_NAME}
  strategy:
    type: Recreate
  template:
    metadata:
      labels:
        app: ${GDCS_WORKLOAD_NAME}
    spec:
      containers:
        - envFrom:
            - configMapRef:
                name: ${GDCS_WORKLOAD_NAME}
          image: ${GDCS_HARBOR_PROJECT_URI}/ollama/ollama:${GDCS_OLLAMA_IMAGE_TAG}
          imagePullPolicy: IfNotPresent
          lifecycle:
            postStart:
              exec:
                command:
                  - /bin/sh
                  - -c
                  - |
                    echo "Waiting for local API to respond..."
                    timeout=60
                    until /bin/ollama list > /dev/null 2>&1 || [ "\$timeout" -le 0 ]; do
                      sleep 1
                      timeout=\$((timeout - 1))
                    done
                    if [ "\$timeout" -le 0 ]; then
                      echo "Timed out waiting for Ollama server to start." >&2
                      exit 1
                    fi
                    echo "Preloading model into memory..."
                    /bin/ollama run "${GDCS_OLLAMA_MODEL}" ""
          livenessProbe:
            failureThreshold: 3
            httpGet:
              path: /
              port: 11434
            periodSeconds: 30
            timeoutSeconds: 10
          name: ollama
          ports:
            - containerPort: 11434
              name: http-ollama
          readinessProbe:
            failureThreshold: 3
            httpGet:
              path: /api/tags
              port: 11434
            periodSeconds: 10
            successThreshold: 1
            timeoutSeconds: 5
          resources:
            limits:
              cpu: ${GDCS_WORKLOAD_CPU_LIMIT}
              memory: ${GDCS_WORKLOAD_MEMORY_LIMIT}
              ${GDCS_WORKLOAD_ACCELERATOR_TYPE}: "${GDCS_WORKLOAD_ACCELERATOR_COUNT}"
            requests:
              cpu: ${GDCS_WORKLOAD_CPU_REQUEST}
              memory: ${GDCS_WORKLOAD_MEMORY_REQUEST}
              ${GDCS_WORKLOAD_ACCELERATOR_TYPE}: "${GDCS_WORKLOAD_ACCELERATOR_COUNT}"
          securityContext:
            allowPrivilegeEscalation: false
            seLinuxOptions:
              type: spc_t
          startupProbe:
            failureThreshold: 60
            httpGet:
              path: /
              port: 11434
            periodSeconds: 10
            timeoutSeconds: 5
          volumeMounts:
            - mountPath: ${GDCS_OLLAMA_PV_MOUNT_PATH}
              name: model-cache
            - mountPath: /dev/shm
              name: shm
      imagePullSecrets:
        - name: ${GDCS_HARBOR_K8S_PULL_SECRET}
      volumes:
        - name: model-cache
          persistentVolumeClaim:
            claimName: ${GDCS_WORKLOAD_NAME}-rwo
        - emptyDir:
            medium: Memory
            sizeLimit: "8Gi"
          name: shm
---
apiVersion: v1
kind: Service
metadata:
  name: ${GDCS_WORKLOAD_NAME}
spec:
  ports:
    - name: http-ollama
      port: 80
      protocol: TCP
      targetPort: 11434
  selector:
    app: ${GDCS_WORKLOAD_NAME}
  type: LoadBalancer
EOF

Terapkan manifes untuk Deployment Ollama:

kubectl apply --filename="${GDCS_USER_GUIDE_HOME}/${GDCS_WORKLOAD_NAME}-deployment.yaml" --namespace="${GDCS_WORKLOAD_NAMESPACE}"

Buat manifes untuk NetworkPolicy Ollama:

cat <<EOF > "${GDCS_USER_GUIDE_HOME}/ollama-netpol.yaml" && echo "Successfully created." || echo "Failed to create!"
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: allow-ollama-ingress
  namespace: ${GDCS_WORKLOAD_NAMESPACE}
spec:
  ingress:
  - from:
    - ipBlock:
        cidr: 0.0.0.0/0
    - namespaceSelector: {}
    ports:
    - port: 11434
      protocol: TCP
  podSelector: {}
  policyTypes:
  - Ingress
EOF

Terapkan manifes untuk NetworkPolicy Ollama:

kubectl apply --filename="${GDCS_USER_GUIDE_HOME}/ollama-netpol.yaml"

Tunggu hingga Pod Berjalan:

kubectl wait \
--for=jsonpath='{.status.phase}'=Running \
"pod" \
--selector="app=${GDCS_WORKLOAD_NAME}" \
--namespace="${GDCS_WORKLOAD_NAMESPACE}" \
--timeout=300s

Tunggu hingga Deployment siap:

kubectl wait \
--for=condition=Available \
"deployment/${GDCS_WORKLOAD_NAME}" \
--namespace="${GDCS_WORKLOAD_NAMESPACE}" \
--timeout=600s

Opsional: Alirkan log container untuk memantau progres startup:

kubectl logs deployment/${GDCS_WORKLOAD_NAME} \
--follow \
--namespace="${GDCS_WORKLOAD_NAMESPACE}"

Validasi

Ambil alamat IP LoadBalancer untuk Ollama:

export OLLAMA_LB_IP=$(kubectl get service/${GDCS_WORKLOAD_NAME} --namespace="${GDCS_WORKLOAD_NAMESPACE}" --output=jsonpath='{.status.loadBalancer.ingress[*].ip}')
echo "OLLAMA_LB_IP=${OLLAMA_LB_IP}"

Kirim permintaan pengujian ke Ollama menggunakan load balancer (Ollama API, pembuatan teks):

curl http://${OLLAMA_LB_IP}/api/generate \
-H "Content-Type: application/json" \
-d '{
  "model": "'${GDCS_OLLAMA_MODEL}'",
  "prompt": "Explain Google Distributed Cloud air-gapped briefly.",
  "stream": false
}'

Kirim permintaan pengujian streaming:

curl http://${OLLAMA_LB_IP}/api/generate \
-H "Content-Type: application/json" \
-d '{
  "model": "'${GDCS_OLLAMA_MODEL}'",
  "prompt": "Explain Google Distributed Cloud air-gapped briefly.",
  "stream": true
}'

Kirim permintaan pengujian ke API Ollama yang kompatibel dengan OpenAI menggunakan load balancer:

curl http://${OLLAMA_LB_IP}/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{
  "model": "'${GDCS_OLLAMA_MODEL}'",
  "messages": [
    {"role": "user", "content": "Explain Google Distributed Cloud air-gapped briefly."}
  ]
}'

Kirim permintaan pengujian ke Ollama menggunakan penerusan port:
```
kubectl port-forward service/${GDCS_WORKLOAD_NAME} \
--namespace="${GDCS_WORKLOAD_NAMESPACE}" 11434:80 &
PF_PID=$!

sleep 1

curl http://127.0.0.1:11434/api/tags

kill ${PF_PID}
```
Catatan: Gemma 4 adalah model penalaran. Pada endpoint yang kompatibel dengan OpenAI, penalaran ditampilkan di choices[0].message.reasoning dan dihitung terhadap max_tokens; dengan anggaran kecil, content dapat tetap kosong. Gunakan endpoint /api/chat khusus Ollama dengan "think": false, atau max_tokens yang lebih besar, saat jawaban yang terlihat diperlukan.

Panduan pengguna model Gemma dengan Ollama di GDC dengan air gap Tetap teratur dengan koleksi Simpan dan kategorikan konten berdasarkan preferensi Anda.

Model yang didukung

Arsitektur

Sebelum memulai

Identity and Access Management

Workstation

Model

Gemma 4 E2B

NVIDIA A100 40 GB

NVIDIA A100 80 GB

NVIDIA H100 94 GB

NVIDIA H200 141 GB

NVIDIA B300 288 GB

Gemma 4 E4B

NVIDIA A100 40 GB

NVIDIA A100 80 GB

NVIDIA H100 94 GB

NVIDIA H200 141 GB

NVIDIA B300 288 GB

Gemma 4 12B

NVIDIA A100 40 GB

NVIDIA A100 80 GB

NVIDIA H100 94 GB

NVIDIA H200 141 GB

NVIDIA B300 288 GB

Gemma 4 26B-A4B

NVIDIA A100 40 GB

NVIDIA A100 80 GB

NVIDIA H100 94 GB

NVIDIA H200 141 GB

NVIDIA B300 288 GB

Gemma 4 31B

NVIDIA A100 40 GB

NVIDIA A100 80 GB

NVIDIA H100 94 GB

NVIDIA H200 141 GB

NVIDIA B300 288 GB

Cluster

Penerapan

Namespace

Bobot model

Server inferensi

Validasi

Panduan pengguna model Gemma dengan Ollama di GDC dengan air gap