Google menggunakan teknologi AI untuk menerjemahkan konten ke dalam bahasa pilihan Anda. Terjemahan AI mungkin mengandung kesalahan.

Implementasi referensi Model Open Weight dengan Ollama di GDC dengan air gap

Dokumen ini memberikan petunjuk langkah demi langkah untuk men-deploy model open-weight seperti Gemma, Llama, dan DeepSeek di lingkungan Google Distributed Cloud (GDC) dengan air gap. Codelab ini membahas penggunaan Ollama untuk kemudahan penggunaan, menggunakan kemampuan platform GDC, termasuk Kubernetes, Harbor, dan resource akselerator.

Arsitektur

Solusinya melibatkan deployment Ollama sebagai backend inferensi dalam container (Deployment) dalam cluster standar. Bobot model disimpan di Persistent Volume yang ukurannya disesuaikan secara dinamis dari direktori model yang didownload dengan overhead 25%, dikonfigurasi dengan kebijakan QoS penyimpanan system-performance-<org>-admin (30 IOPS per GiB dengan batas minimum 6.148 IOPS, atau performance-rwo jika penyimpanan blok berperforma tinggi diaktifkan), dan diisi menggunakan alur kerja pemuat model. Service Kubernetes berjenis LoadBalancer mengekspos API backend, dan objek NetworkPolicy / ProjectNetworkPolicy mengamankan akses ke API tersebut.

Model Bobot Terbuka dengan arsitektur referensi Ollama di GDC dengan air gap.

Sebelum memulai

Sebelum melanjutkan deployment, pastikan lingkungan Anda memenuhi semua prasyarat yang diperlukan dan utilitas command line yang diperlukan telah dikonfigurasi dengan benar. Menyiapkan alat ini di workstation Anda sangat penting untuk mengelola container registry, berinteraksi dengan cluster, dan mengotomatiskan proses deployment.

Cluster Standard yang dibuat dengan resource yang memadai (CPU, Memori, akselerator). Persyaratan akselerator minimum akan bervariasi bergantung pada model.
Instance Harbor tersedia dan dapat diakses.
IAM yang diperlukan telah diterapkan.
Workstation dengan konektivitas yang diperlukan ke lingkungan dan internet

Konfigurasi lingkungan

Identity and Access Management

Pastikan akun, peran, dan izin IAM yang diperlukan telah dikonfigurasi dengan benar.

Peran GDC User di project (RoleBinding di namespace project, yang diberikan oleh Project IAM Admin):

Harbor Instance Viewer (harbor-instance-viewer)
Harbor Project Creator (harbor-project-creator, hanya jika project Harbor belum ada)
Standard Cluster Admin (standard-cluster-admin, diperlukan untuk gdcloud clusters get-credentials)

Peran GDC User di cluster standar: peran project sebelumnya tidak memberikan izin di dalam cluster. Project IAM Admin juga harus mengikat pengguna ke StandardClusterRole cluster-admin dengan StandardClusterRoleBinding di namespace project di server API pengelolaan; binding ini akan diterapkan ke cluster standar project dalam hitungan detik (status.clusters[].conditions menampilkan Propagated=True).

cat <<EOF | kubectl --kubeconfig MANAGEMENT_API_SERVER apply -f -
apiVersion: iam.gdc.goog/v1
kind: StandardClusterRoleBinding
metadata:
  name: BINDING_NAME
  namespace: PROJECT
spec:
  roleRef:
    apiGroup: iam.gdc.goog
    kind: StandardClusterRole
    name: cluster-admin
  subjects:
    - apiGroup: rbac.authorization.k8s.io
      kind: User
      name: USER
EOF

Ganti kode berikut:

MANAGEMENT_API_SERVER: jalur ke file kubeconfig server API pengelolaan.
BINDING_NAME: nama untuk binding, misalnya alex-cluster-admin (huruf kecil, angka, dan tanda hubung).
PROJECT: nama project.
USER: nama pengguna yang akan diikat, persis seperti yang dilaporkan oleh penyedia identitas Anda (misalnya, alamat email).

Bantuan Operator Infrastruktur (hanya jika performance-rwo tidak tersedia di cluster standar): panduan pengguna membuat model PersistentVolumeClaim dengan StorageClass standard-rwo default, yang membatasi volume model berukuran tepat hingga 3 IOPS per GiB. Minta Operator Infrastruktur Anda untuk menerapkan kebijakan QoS system-performance-<ORG>-admin ke volume yang mendukung model PersistentVolumeClaim agar menerima batas minimum 6.148 IOPS. Panduan pengguna mencetak nilai yang diperlukan Operator Infrastruktur.

Izin akun robot Harbor untuk crane:

Repositori Daftar
Repositori Pull
Repositori Push
Membaca Artefak
Artefak Daftar
Buat Tag
Tag Daftar

Izin akun robot Harbor untuk pengambilan image Kubernetes:

Repositori Daftar
Repositori Pull
Membaca Artefak
Artefak Daftar
Tag Daftar

Workstation

Jalankan semua perintah dalam panduan ini dari workstation dengan alat dan konektivitas yang diperlukan ke lingkungan dan internet, kecuali jika ada langkah yang menyatakan sebaliknya.

Persyaratan

Alat berikut harus diinstal di workstation:

crane: mengelola dan menyalin image container antar-registry (dokumentasi).
gdcloud: antarmuka command line (CLI) untuk mengelola resource GDC (dokumentasi).
kubectl: antarmuka command line (CLI) yang digunakan untuk berkomunikasi dengan dan mengelola cluster Kubernetes.
jq: pemroses JSON command line yang ringan dan fleksibel.
ollama: alat open source yang memungkinkan Anda mendownload dan menjalankan model.
yq: pemroses YAML command line portabel.

Konfigurasi workstation

Informasi berikut tentang lingkungan diperlukan untuk Konfigurasi Workstation:

GDC_STANDARD_CLUSTER_NAME: Nama cluster standar GDC.
GDC_DOMAIN_SUFFIX: Akhiran domain untuk lingkungan GDC (misalnya, gdc.example.com).
GDC_ORG: Nama organisasi GDC.
GDC_PROJECT: Nama project GDC.
GDC_ZONE: Nama zona deployment GDC.
GDC_HARBOR_INSTANCE_NAME: Nama instance Harbor dalam project.
GDCS_HARBOR_PROJECT_NAME: Nama project Harbor yang akan digunakan untuk image (default: solutions).
GDCS_HARBOR_CRANE_ROBOT_NAME: Nama akun robot crane Harbor.
GDCS_HARBOR_CRANE_ROBOT_TOKEN: Token autentikasi untuk akun robot crane Harbor.
GDCS_HARBOR_K8S_ROBOT_NAME: Nama akun robot penarikan image Kubernetes Harbor.
GDCS_HARBOR_K8S_ROBOT_TOKEN: Token autentikasi untuk akun robot penarikan image Kubernetes Harbor.

Setelah Anda mengumpulkan nilai untuk semua variabel yang diperlukan, lanjutkan dengan membuat file variabel lingkungan. Setelah dibuat, Anda dapat mengedit file secara manual kapan saja.

Buat direktori solusi root dan folder rahasia:

mkdir -p ${HOME}/gdcag-solutions/env.d
mkdir -p ${HOME}/gdcag-solutions/secrets

touch ${HOME}/gdcag-solutions/secrets/harbor_crane_robot_token
touch ${HOME}/gdcag-solutions/secrets/harbor_k8s_robot_token

chmod u=rwx,go= ${HOME}/gdcag-solutions/secrets
chmod -R u=rw,go= ${HOME}/gdcag-solutions/secrets/*

Buat file konfigurasi lingkungan platform:

cat << 'EOF' > ${HOME}/gdcag-solutions/env.d/platform.sh && echo "Successfully created." || echo "Failed to create!"
# Infrastructure (Platform Native)
export GDC_STANDARD_CLUSTER_NAME="STANDARD_CLUSTER_NAME"
export GDC_DOMAIN_SUFFIX="DOMAIN_SUFFIX"
export GDC_ORG="ORG"
export GDC_PROJECT="PROJECT"
export GDC_ZONE="ZONE"
export GDC_HARBOR_INSTANCE_NAME="HARBOR_INSTANCE_NAME"

# Derived platform values
export GDC_ZONAL_HOSTNAME="${GDC_ORG}.${GDC_ZONE}.${GDC_DOMAIN_SUFFIX}"
export GDC_ZONAL_CONSOLE_URL="https://console.${GDC_ZONAL_HOSTNAME}"
export GDC_HARBOR_HOST="${GDC_HARBOR_INSTANCE_NAME}-${GDC_PROJECT}.${GDC_ORG}.${GDC_ZONE}.${GDC_DOMAIN_SUFFIX}"
EOF

Ganti kode berikut:

STANDARD_CLUSTER_NAME: Nama cluster standar GDC.
DOMAIN_SUFFIX: Akhiran domain GDC.
ORG: Organisasi GDC.
PROJECT: Project GDC.
ZONE: Zona GDC.
HARBOR_INSTANCE_NAME: Nama instance GDC Harbor.

Buat file konfigurasi lingkungan registry:

cat << 'EOF' > ${HOME}/gdcag-solutions/env.d/registry.sh && echo "Successfully created." || echo "Failed to create!"
# GDC Solutions Registry & Secrets
export GDCS_HARBOR_PROJECT_NAME="solutions"
export GDCS_HARBOR_CRANE_ROBOT_NAME='HARBOR_CRANE_ROBOT_NAME'
export GDCS_HARBOR_CRANE_ROBOT_TOKEN="$(cat ${GDCS_ROOT_HOME}/secrets/harbor_crane_robot_token)"
export GDCS_HARBOR_K8S_ROBOT_NAME='HARBOR_K8S_ROBOT_NAME'
export GDCS_HARBOR_K8S_ROBOT_TOKEN="$(cat ${GDCS_ROOT_HOME}/secrets/harbor_k8s_robot_token)"
export GDCS_HARBOR_K8S_PULL_SECRET="gdcs-image-pull-secret"

# Derived registry values
export GDCS_HARBOR_PROJECT_URI="${GDC_HARBOR_HOST}/${GDCS_HARBOR_PROJECT_NAME}"
export GDCS_HARBOR_CHART_OCI_URI="oci://${GDCS_HARBOR_PROJECT_URI}"
EOF

Ganti kode berikut:

HARBOR_CRANE_ROBOT_NAME: nama lengkap akun robot Harbor untuk crane, seperti yang ditampilkan Harbor, termasuk awalan robot$. Pertahankan tanda kutip tunggal agar shell tidak memperluas $.
HARBOR_K8S_ROBOT_NAME: nama lengkap akun robot Harbor untuk penarikan image Kubernetes, seperti yang ditampilkan Harbor, termasuk awalan robot$. Pertahankan tanda kutip tunggal agar shell tidak memperluas $.

Tambahkan token Anda ke file secret:

set +o history

echo "CRANE_ROBOT_TOKEN" > ${HOME}/gdcag-solutions/secrets/harbor_crane_robot_token
echo "KUBERNETES_ROBOT_TOKEN" > ${HOME}/gdcag-solutions/secrets/harbor_k8s_robot_token

set -o history

Ganti kode berikut:

CRANE_ROBOT_TOKEN: secret akun robot crane.
KUBERNETES_ROBOT_TOKEN: secret akun robot penarikan image Kubernetes.

Buat file loader lingkungan root:

cat << 'EOF' > ${HOME}/gdcag-solutions/env.sh && echo "Successfully created." || echo "Failed to create!"
export GDCS_ROOT_HOME="${HOME}/gdcag-solutions"
echo "GDCS_ROOT_HOME=${GDCS_ROOT_HOME}"

# Sourced in dependency order
source "${GDCS_ROOT_HOME}/env.d/platform.sh"
source "${GDCS_ROOT_HOME}/env.d/registry.sh"
EOF

Mengonfigurasi variabel solusi

Buat direktori implementasi solusi:

mkdir -p ${HOME}/gdcag-solutions/open-weight-models/ollama/env.d

Buat file konfigurasi akselerator:
```
cat << 'EOF' > ${HOME}/gdcag-solutions/open-weight-models/ollama/env.d/accelerator.sh && echo "Successfully created." || echo "Failed to create!"
# Workload Accelerator (NVIDIA GPU)
export GDCS_WORKLOAD_ACCELERATOR_TYPE="WORKLOAD_ACCELERATOR_TYPE"
export GDCS_WORKLOAD_ACCELERATOR_COUNT="1"
EOF
```
Ganti kode berikut:
- WORKLOAD_ACCELERATOR_TYPE: nama resource GPU yang diekspos oleh node GPU Anda, misalnya nvidia.com/gpu-pod-NVIDIA_A100_80GB_PCIE. Panduan pengguna mencantumkan nama resource setiap GPU yang ditawarkan GDC dengan air gap. Setelah mengambil kredensial cluster, Anda dapat mencantumkan nama di cluster dengan kubectl get nodes --output=json | jq -r '[.items[].status.allocatable | keys[] | select(startswith("nvidia.com/"))] | unique[]'.

Buat file konfigurasi lingkungan solusi:

cat << 'EOF' > ${HOME}/gdcag-solutions/open-weight-models/ollama/env.d/ollama.sh && echo "Successfully created." || echo "Failed to create!"
# Images
export GDCS_BUSYBOX_IMAGE_TAG="1.38.0"
export GDCS_OLLAMA_IMAGE_TAG="0.32.14"

# Docker configuration directories for crane and Kubernetes
export GDCS_HARBOR_CRANE_DOCKER_CONFIG="${GDCS_IMPLEMENTATION_HOME}/docker/crane"
export GDCS_HARBOR_K8S_DOCKER_CONFIG="${GDCS_IMPLEMENTATION_HOME}/docker/k8s"
EOF

Buat file loader lingkungan implementasi:

cat << 'EOF' > ${HOME}/gdcag-solutions/open-weight-models/ollama/env.sh && echo "Successfully created." || echo "Failed to create!"
source "${HOME}/gdcag-solutions/env.sh"

export GDCS_IMPLEMENTATION_HOME="${HOME}/gdcag-solutions/open-weight-models/ollama"
echo "GDCS_IMPLEMENTATION_HOME=${GDCS_IMPLEMENTATION_HOME}"

# Sourced in dependency order
source "${GDCS_IMPLEMENTATION_HOME}/env.d/accelerator.sh"
source "${GDCS_IMPLEMENTATION_HOME}/env.d/ollama.sh"
EOF

Edit dan tinjau file lingkungan dengan editor pilihan Anda:

${EDITOR:-vi} ${HOME}/gdcag-solutions/env.d/platform.sh
${EDITOR:-vi} ${HOME}/gdcag-solutions/env.d/registry.sh
${EDITOR:-vi} ${HOME}/gdcag-solutions/open-weight-models/ollama/env.d/accelerator.sh
${EDITOR:-vi} ${HOME}/gdcag-solutions/open-weight-models/ollama/env.d/ollama.sh

Ambil file lingkungan:

source ${HOME}/gdcag-solutions/open-weight-models/ollama/env.sh

Outputnya mirip dengan hal berikut ini:

GDCS_ROOT_HOME=HOME_DIRECTORY_PATH/gdcag-solutions
GDCS_IMPLEMENTATION_HOME=HOME_DIRECTORY_PATH/gdcag-solutions/open-weight-models/ollama

GDC

Panduan ini mengasumsikan bahwa workstation Anda dikonfigurasi untuk memercayai sertifikat TLS untuk lingkungan GDC dan instance Harbor Anda.

Konfigurasikan gdcloud:

gdcloud config set core/account "default-user"
gdcloud config set core/organization_console_url "${GDC_ZONAL_CONSOLE_URL}"
gdcloud config set core/project "${GDC_PROJECT}"
gdcloud config set core/zone "${GDC_ZONE}"

Lakukan autentikasi ke lingkungan GDC:
```
gdcloud auth login
```

Cluster

Ambil kredensial cluster:

gdcloud clusters get-credentials "${GDC_STANDARD_CLUSTER_NAME}" \
--project="${GDC_PROJECT}" \
--standard \
--zone="${GDC_ZONE}"

Verifikasi konektivitas ke cluster:

kubectl get nodes -L node.cluster.private.gdc.goog/machine-class

Persiapan migrasi artefak

Memverifikasi konektivitas dan konfigurasi untuk artifact registry.
- Mengonfigurasi Docker untuk memercayai root CA Harbor | Google Distributed Cloud dengan air gap

Buat file konfigurasi Docker untuk crane. Akun robot digunakan untuk mendorong lapisan image besar guna menghindari waktu tunggu token autentikasi saat menggunakan helper kredensial Managed Harbor Service (MHS) (docker-credential-mhs) dengan akun pengguna:
```
export DOCKER_CONFIG="${GDCS_HARBOR_CRANE_DOCKER_CONFIG}"

crane auth login "${GDC_HARBOR_HOST}" \
--password="${GDCS_HARBOR_CRANE_ROBOT_TOKEN}" \
--username="${GDCS_HARBOR_CRANE_ROBOT_NAME}"
```

Buat file konfigurasi Docker untuk Kubernetes:

export DOCKER_CONFIG="${GDCS_HARBOR_K8S_DOCKER_CONFIG}"

crane auth login "${GDC_HARBOR_HOST}" \
--password="${GDCS_HARBOR_K8S_ROBOT_TOKEN}" \
--username="${GDCS_HARBOR_K8S_ROBOT_NAME}"

Buat skrip seed_registry.sh:

cat << 'EOF' > ${GDCS_IMPLEMENTATION_HOME}/seed_registry.sh && echo "Successfully created." || echo "Failed to create!"
#!/bin/bash

# seed_registry.sh: Modular artifact migration for GDC Solutions

# Requires the env.sh file to be sourced first.
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"

set -o nounset

source "${SCRIPT_DIR}/env.sh"

# Set the Docker config
export DOCKER_CONFIG="${GDCS_HARBOR_CRANE_DOCKER_CONFIG}"

eval "${SERIALIZED_IMAGES}"

# Ensure GDCS_REGISTRY_IMAGES is set
if [[ ${#GDCS_REGISTRY_IMAGES[@]} -eq 0 ]]; then
  echo "GDCS_REGISTRY_IMAGES must be set, exiting..."
  exit 1
fi

# Migrate the images
for source_image in "${GDCS_REGISTRY_IMAGES[@]}"; do
  # Strip the registry host only when the first path segment is a host (contains a dot or a port)
  first_segment="${source_image%%/*}"
  if [[ "${first_segment}" == *.* || "${first_segment}" == *:* ]]; then
    image_path="${source_image#*/}"
  else
    image_path="${source_image}"
  fi
  destination_image="${GDCS_HARBOR_PROJECT_URI}/${image_path}"

  # Ensure the folder structure is created
  crane append \
  --new_layer=<(tar czf - -T /dev/null) \
  --new_tag="${destination_image%:*}:create" \
  --oci-empty-base 2> /dev/null || true

  # Copy the linux/amd64 platform only to avoid transferring multi-arch layers over air-gapped links
  crane copy --platform linux/amd64 "${source_image}" "${destination_image}" || { echo "Failed to copy ${source_image} to ${destination_image}"; exit 1; }
done

echo "Migration complete: Images are available at ${GDCS_HARBOR_PROJECT_URI}"
EOF
chmod u+x "${GDCS_IMPLEMENTATION_HOME}/seed_registry.sh"

Tentukan daftar image container yang diperlukan untuk solusi:

declare -a GDCS_REGISTRY_IMAGES=(
  "docker.io/library/busybox:${GDCS_BUSYBOX_IMAGE_TAG}"
  "docker.io/ollama/ollama:${GDCS_OLLAMA_IMAGE_TAG}"
)
export SERIALIZED_IMAGES=$(declare -p GDCS_REGISTRY_IMAGES)

Lakukan seeding image container yang diperlukan ke artifact registry:
```
${GDCS_IMPLEMENTATION_HOME}/seed_registry.sh
```

Persiapan model

Buat skrip ollama_model_weight_download.sh. Perintah ini menjalankan ollama serve lokal sementara dengan OLLAMA_MODELS yang diarahkan ke direktori download dan menarik model ke dalamnya:

cat << 'EOF' > ${GDCS_IMPLEMENTATION_HOME}/ollama_model_weight_download.sh && echo "Successfully created." || echo "Failed to create!"
#!/bin/bash

# ollama_model_weight_download.sh: pull GDCS_OLLAMA_MODEL into GDCS_OLLAMA_MODEL_DOWNLOAD_DIR with a temporary local Ollama server.

# Requires: GDCS_OLLAMA_MODEL, GDCS_OLLAMA_MODEL_DOWNLOAD_DIR
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"

set -o nounset
set -o pipefail

# Ensure GDCS_OLLAMA_MODEL_DOWNLOAD_DIR is set
if [[ -z "${GDCS_OLLAMA_MODEL_DOWNLOAD_DIR:-}" ]]; then
  echo "GDCS_OLLAMA_MODEL_DOWNLOAD_DIR must be set, exiting..."
  exit 1
fi

mkdir -p "${GDCS_OLLAMA_MODEL_DOWNLOAD_DIR}"

# Use a dedicated local port so an already running Ollama instance is not reused
export OLLAMA_HOST="${OLLAMA_DOWNLOAD_HOST:-127.0.0.1:11435}"
export OLLAMA_MODELS="${GDCS_OLLAMA_MODEL_DOWNLOAD_DIR}"
ollama serve > "${GDCS_OLLAMA_MODEL_DOWNLOAD_DIR}/ollama-serve.log" 2>&1 &
OLLAMA_PID=$!

# Set up a trap to kill ollama serve when the script exits
trap "kill ${OLLAMA_PID} 2>/dev/null" EXIT INT TERM

# Wait for the local server
for i in $(seq 1 30); do
  ollama list > /dev/null 2>&1 && break
  sleep 1
done

ollama pull "${GDCS_OLLAMA_MODEL}" || exit 1

rm -f "${GDCS_OLLAMA_MODEL_DOWNLOAD_DIR}/ollama-serve.log"
echo "Model '${GDCS_OLLAMA_MODEL}' is available in '${GDCS_OLLAMA_MODEL_DOWNLOAD_DIR}'."
EOF
chmod u+x "${GDCS_IMPLEMENTATION_HOME}/ollama_model_weight_download.sh"

Buat skrip model_weight_upload.sh. Skrip ini menyalin direktori model lokal ke PersistentVolume melalui pemuat model Pod (streaming tar GNU, termasuk file yang lebih besar dari 8 GiB) dan melewati file yang sudah ada dengan ukuran yang sama:

cat << 'EOF' > ${GDCS_IMPLEMENTATION_HOME}/model_weight_upload.sh && echo "Successfully created." || echo "Failed to create!"
#!/bin/bash

# model_weight_upload.sh: copy LOCAL_MODEL_DIR into REMOTE_MODEL_DIR on the PersistentVolume mounted in the loader Pod.

# Requires: GDCS_WORKLOAD_NAMESPACE, GDCS_MODEL_LOADER_POD_NAME, LOCAL_MODEL_DIR, REMOTE_MODEL_DIR

# Optional: REMOTE_MOUNT_DIR (PersistentVolume mount path, defaults to REMOTE_MODEL_DIR) - its lost+found directory is removed.
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"

set -o errexit
set -o nounset
set -o pipefail

# Ensure LOCAL_MODEL_DIR is set
if [[ -z "${LOCAL_MODEL_DIR:-}" ]]; then
  echo "LOCAL_MODEL_DIR must be set, exiting..."
  exit 1
fi

# Ensure REMOTE_MODEL_DIR is set
if [[ -z "${REMOTE_MODEL_DIR:-}" ]]; then
  echo "REMOTE_MODEL_DIR must be set, exiting..."
  exit 1
fi

REMOTE_MOUNT_DIR="${REMOTE_MOUNT_DIR:-${REMOTE_MODEL_DIR}}"

KEEPALIVE_INTERVAL_SEC=300
start_session_keepalive() {
  echo "[KeepAlive] Started background heartbeat (every ${KEEPALIVE_INTERVAL_SEC}s)."
  while true; do
    sleep "${KEEPALIVE_INTERVAL_SEC}"
    # Lightweight API request to prevent session inactivity logout
    if kubectl get namespace "${GDCS_WORKLOAD_NAMESPACE}" >/dev/null 2>&1; then
      echo "[KeepAlive] Session heartbeat sent at $(date)"
    else
      echo "[KeepAlive] Heartbeat failed or context expired at $(date)"
    fi
  done
}

start_session_keepalive &
MY_KEEPALIVE_PID=$!

cleanup() {
  echo "[Cleanup] Stopping keep-alive process (PID: ${MY_KEEPALIVE_PID})..."
  kill "${MY_KEEPALIVE_PID}" 2>/dev/null || true
  wait "${MY_KEEPALIVE_PID}" 2>/dev/null || true
  echo "[Cleanup] Session keep-alive terminated."
}

trap cleanup EXIT INT TERM

# Remove the lost+found directory
kubectl exec "pod/${GDCS_MODEL_LOADER_POD_NAME}" \
--namespace="${GDCS_WORKLOAD_NAMESPACE}" \
-- rm -rf "${REMOTE_MOUNT_DIR}/lost+found"

# Create the model directory
kubectl exec "pod/${GDCS_MODEL_LOADER_POD_NAME}" \
--namespace="${GDCS_WORKLOAD_NAMESPACE}" \
-- mkdir -p "${REMOTE_MODEL_DIR}"

# Loop through all files in the local directory
FAILED_FILES=()
MAX_UPLOAD_ATTEMPTS="${MAX_UPLOAD_ATTEMPTS:-3}"
remote_size_of() {
    # Prints the remote file size (0 if the file does not exist)
    kubectl exec "pod/${GDCS_MODEL_LOADER_POD_NAME}" --namespace="${GDCS_WORKLOAD_NAMESPACE}" -- stat -c%s "$1" 2>/dev/null || echo 0
}

# Loop through all files in the local directory (process substitution keeps FAILED_FILES in this shell)
while read -r local_file; do
    # Get relative path
    rel_path="${local_file#"${LOCAL_MODEL_DIR}"/}"
    remote_file="${REMOTE_MODEL_DIR}/${rel_path}"

    # Get local file size
    local_size=$(stat -L -c%s "${local_file}")

    # Get remote file size (returns 0 if file does not exist)
    remote_size=$(remote_size_of "${remote_file}")

    if [ "${local_size}" -eq "${remote_size}" ]; then
        echo "[Loader] Skipping: ${rel_path} (Size matches)"
        continue
    fi
    if [ "${remote_size}" -ne "0" ]; then
        echo "[Loader] Mismatch: ${rel_path} (${remote_size} remote vs ${local_size} local)"
    fi

    # Ensure remote parent subdirectories exist if your model has folders
    remote_parent=$(dirname "${remote_file}")
    kubectl exec "pod/${GDCS_MODEL_LOADER_POD_NAME}" \
    --namespace="${GDCS_WORKLOAD_NAMESPACE}" \
    -- mkdir -p "${remote_parent}"

    attempt=1
    while [ "${attempt}" -le "${MAX_UPLOAD_ATTEMPTS}" ]; do
        echo "[Loader] Uploading: ${rel_path} (${local_size} bytes, attempt ${attempt}/${MAX_UPLOAD_ATTEMPTS})..."
        # Copy the individual file using GNU tar streaming.
        # Note: 'kubectl cp' uses Go's archive/tar, which defaults to PAX format for files > 8 GiB.
        # BusyBox's tar in the loader pod does not parse PAX size headers and fails with 'tar: invalid tar magic'.
        # Streaming via GNU tar (--format=gnu) uses base-256 binary encoding for files > 8 GiB, which BusyBox tar supports.
        tar --format=gnu -C "$(dirname "${local_file}")" -cf - "$(basename "${local_file}")" | \
        kubectl exec -i "pod/${GDCS_MODEL_LOADER_POD_NAME}" \
        --namespace="${GDCS_WORKLOAD_NAMESPACE}" \
        -- tar -xf - -C "${remote_parent}" || true

        # Verify: an interrupted exec stream leaves a truncated file behind without an error
        remote_size=$(remote_size_of "${remote_file}")
        if [ "${local_size}" -eq "${remote_size}" ]; then
            echo "[Loader] Verified: ${rel_path} (${remote_size} bytes)"
            break
        fi
        echo "[Loader] Incomplete upload: ${rel_path} (${remote_size} remote vs ${local_size} local), retrying..."
        attempt=$((attempt + 1))
    done
    if [ "${local_size}" -ne "${remote_size}" ]; then
        FAILED_FILES+=("${rel_path}")
    fi
done < <(find -L "${LOCAL_MODEL_DIR}" -type f)

if [ "${#FAILED_FILES[@]}" -ne 0 ]; then
    echo "[Loader] FAILED: ${#FAILED_FILES[@]} file(s) could not be uploaded completely after ${MAX_UPLOAD_ATTEMPTS} attempts:"
    printf '[Loader]   %s\n' "${FAILED_FILES[@]}"
    exit 1
fi
echo "[Loader] Completed"

# Fix file permissions
kubectl exec "pod/${GDCS_MODEL_LOADER_POD_NAME}" \
--namespace="${GDCS_WORKLOAD_NAMESPACE}" \
-- chown 0:0 -R "${REMOTE_MODEL_DIR}"

# Flush the volume before the loader Pod is deleted (a detached volume with unwritten page cache loses data)
kubectl exec "pod/${GDCS_MODEL_LOADER_POD_NAME}" \
--namespace="${GDCS_WORKLOAD_NAMESPACE}" \
-- sync
EOF
chmod u+x "${GDCS_IMPLEMENTATION_HOME}/model_weight_upload.sh"

Men-deploy model

Panduan pengguna model Gemma dengan Ollama: Gemma 4 E2B, E4B, 12B, 26B-A4B, dan 31B, dengan opsi GPU GDC dengan air gap

Implementasi referensi Model Open Weight dengan Ollama di GDC dengan air gap Tetap teratur dengan koleksi Simpan dan kategorikan konten berdasarkan preferensi Anda.

Arsitektur

Sebelum memulai

Konfigurasi lingkungan

Identity and Access Management

Workstation

Persyaratan

Konfigurasi workstation

Mengonfigurasi variabel solusi

GDC

Cluster

Persiapan migrasi artefak

Persiapan model

Men-deploy model

Implementasi referensi Model Open Weight dengan Ollama di GDC dengan air gap