In a standard GDC deployment, automated tools attempt to
convert uploaded images and inject the gdc-guest-agent. However, for
unsupported distributions or OSs with immutable (read-only) filesystems, this
automated translation fails, breaking the image import process and preventing
network and SSH access. This architecture solves that problem by detailing how
to perform offline image surgery to manually configure the OS, bypass the
platform's automated translation engine, and optionally create a repeatable
"Golden Image" for scalable deployments. This document is intended for Platform
Administrators (PA) and Application Operators (AO).
Features and capabilities
- Offline image surgery: Mount a virtual disk offline to satisfy GDC air-gapped requirements (such as internal NTP synchronization and root password configuration) before uploading the image.
- Translation bypass (manual mode): Instruct the GDC platform to skip its standard automated configuration and agent-installation processes, preventing import failures on unsupported or immutable operating systems.
- Direct SSH and serial console access: Replace automated guest agent key injection with serial console access and manual SSH public key injection into the OS.
- Golden image automation: Capture a customized VM disk and convert it
into a standard
VirtualMachineImagetemplate, enabling repeatable deployments that process startup scripts.
Architectural principles
- Workload flexibility and sovereignty: Unblock the migration of legacy, specialized, or high-security workloads by deploying custom Linux distributions.
- Separation of concerns: Keep OS image preparation separate from platform infrastructure. Shifting configuration to a local workstation avoids corrupting image imports with incompatible automated platform scripts.
- Resilience and reliability: Air-gapped environments are sensitive to time drift. This architecture configures internal platform NTP servers directly in the image's core configuration so certificates and logging function immediately upon boot.
- Scalability through golden images: Avoid performing manual image surgery on every deployed VM by using an initial preparation VM to generate a reusable artifact.
Architecture

1. Image preparation (local workstation)
- Mount Image: Map the
.qcow2virtual disk to a physical block device usingqemu-nbd. - Chroot Entry: Access the disk's internal filesystem via
chrootto perform image changes. - Hardcode Configs: Manually set internal NTP servers in
/etc/chrony.confand establish a temporary root password. - Verify Drivers: Ensure necessary Virtio drivers are baked into the kernel for disk and network compatibility.
- Install Agent if needed: Install the GDC guest agent to let GDC manage the guests SSH access.
2. GDC import (manual mode)
- Translation bypass: Import the image by using the gdcloud CLI
with the
--os=garden-linuxflag. - Skip automation: This flag acts as a manual mode switch, instructing the platform to skip automated configuration and guest agent injection that can fail on unsupported distributions.
3. Network and access management
- IP address and firewall: Deploy the VM and configure
VirtualMachineExternalAccessfor an ingress IP address and aProjectNetworkPolicy(PNP) to open SSH port 22. - Initial sign-in: Access the VM through the GDC serial console by using the root password set during preparation.
Manual SSH setup: Add the administrator's public SSH key to the
~/.ssh/authorized_keysfile.4. Golden image capture pipeline
VM Capture: Customize the running "preparation VM" and shut it down cleanly to ensure data consistency.
Template Creation: Convert the boot disk into a standard
VirtualMachineImageartifact.Scalability: Use the resulting Golden Image for repeatable, large-scale deployments that natively support platform startup scripts and GDC-managed user/SSH access.
Concepts and technologies
This section breaks down the solution into distinct, functional components required to support custom image imports.
- Image Preparation Unit (Local Workstation):
- Responsibilities: Modifies the sparse virtual disk image (
.qcow2) before it enters the GDC platform, ensuring all drivers and network requirements are met. - Interfaces: Utilizes QEMU utilities. Specifically,
qemu-nbdacts as a bridge to map the virtual disk file to a physical block device (e.g.,/dev/nbd0). Thechrootcommand is then used to change the root directory, allowing administrators to log into the mounted image and execute configurations (like editing/etc/chrony.confor setting passwords) exactly as if they were natively logged in.
- Responsibilities: Modifies the sparse virtual disk image (
- GDC Import Interface:
- Responsibilities: Safely ingests the modified image without triggering the GDC platform's breaking automation.
- Interfaces: The
gdcloud compute images importCLI tool. The critical component here is the--os=garden-linuxflag. Because GDC recognizes "Garden Linux" as an immutable system, this flag acts as a "Manual Mode" switch, instructing the translation engine to skip automated agent installation and image translation entirely.
- Network and Access Management:
- Responsibilities: Facilitates secure communication and administrative
access to the unsupported OS without reliance on the native
gdc-guest-agent. - Interfaces: Relies on configuring a
VirtualMachineExternalAccessresource to allocate an Ingress IP, and a ProjectNetworkPolicy (PNP) to open port 22 for SSH traffic through the project firewall. Initial access is achieved via the GDC Serial Console using a password set during offline surgery, allowing the administrator to manually paste their public SSH key into the~/.ssh/authorized_keysfile.
- Responsibilities: Facilitates secure communication and administrative
access to the unsupported OS without reliance on the native
- Golden Image Capture Pipeline:
- Responsibilities: Converts a manually staged "preparation VM" into a reusable artifact.
- Data Flow: Prepares the running VM (e.g., ensuring
cloud-initis installed to handle KubeVirtcloudInitConfigDrivemetadata), cleanly shuts the VM down to ensure disk consistency, and captures the boot disk into aVirtualMachineImageImportresource.
Considerations
- Scalability and Performance: The initial offline preparation is a manual, highly technical process and is not horizontally scalable. However, converting the first successfully deployed VM into a Golden Image allows the solution to scale infinitely for subsequent deployments across the project.
- Operational Complexity: This is strictly a customer-managed solution.
The operational burden on the Platform Administrator is high during the
initial configuration, requiring root privileges on a Linux dev workstation,
manual dependency resolution for air-gapped package installations (e.g.,
wget,cloud-init), and an understanding of disk partition layouts. - Security and Compliance: The solution requires creating a temporary root password for emergency Serial Console access. Strict operational hygiene must be maintained to rotate or remove this password once public SSH keys are manually injected, ensuring the system transitions to key-based authentication.
- Availability and Reliability (NTP): Air-gapped environments require
precise time synchronization for core services (like certificates and
authentication) to function. The architecture dictates that NTP (e.g.,
chrony) must be strictly configured to GDC's internal time servers (ntp1.org.internal, etc.) during the offline configuration phase.
Design decision
- Context & Constraints: GDC air-gapped native import translators do not currently support immutable filesystems (like RHCOS 9) or certain Linux distributions (like SLES 16 or Debian builds), causing image import failures and preventing basic SSH access.
- Rationale & Trade-offs: The decision was made to utilize a "shift-left"
approach using
qemu-nbdandchrootcombined with the--os=garden-linuxCLI flag. - Impact: This decision may require operators to manage their own SSH keys, OS patching, and network routing manually, but successfully unblocks the migration of mission-critical BYO operating systems for stringent compliance environments.
Assumptions and limitations
Assumptions
- Virtio Driver Compatibility: The target operating system kernel must
already include the required Virtio drivers (
virtio_pcifor the base bus,virtio_blkorvirtio_scsifor disk access, andvirtio_netfor networking). If these drivers are not baked into the OS before import, the VM will fail to boot on GDC's underlying KubeVirt hypervisor because it will not be able to see its own hard drive or network card. - The administrator performing the import has an existing Linux workstation
with
qemu-utilsinstalled,sudoprivileges, and the ability to mount block devices.
Limitations
- Guest Agent Functionality: Automated provisioning of SSH keys and users
via the GDC control plane is unsupported
out-of-the-box because the translation engine is bypassed. Full platform
integration is only possible if the customer manually compiles and
successfully installs the
gdc-guest-agent .debor.rpmpackages during the chroot phase. - Specific Filesystems: Immutability layers like
ostree(used by RHCOS) require specific bind-mounting logic inside the chroot environment that is not applicable to standard root filesystems likeext4orBtrfs(used by Debian or SLES).
What's next
- Debian 11 OS image import reference implementation
- Oracle Linux 8 and 9 OS image import reference implementation
- SLES 16, 15SP7, and 15SP6 OS image import reference implementation