Adapt operating systems for Google Cloud

This document describes how to adapt operating system (OS) images in Google Cloud to use the enterprise OS images that you have licenses for.

Google Cloud provides OS images for Debian, Ubuntu LTS, Rocky Linux, and Container-Optimized OS. If you want to use any of these provided images, then proceed to create an instance and select the image you want to use during instance creation.

If you have a license for an enterprise OS that Google Cloud doesn't provide an OS image for, such as Windows, Red Hat Enterprise Linux (RHEL), or SUSE Linux Enterprise Server (SLES), then use the script provided in this document to adapt an image for use with your license. The script installs the drivers required for you to use your image to create instances in Google Cloud.

Before you begin

  • Install the jq command line processor.
  • If you haven't already done so, upload a disk that contains the OS image that you want to use to a Cloud Storage bucket.
  • Confirm that you have the required permissions to create custom images.
  • If you haven't already, then set up authentication. Authentication is the process by which your identity is verified for access to Google Cloud services and APIs. To run code or samples from a local development environment, you can authenticate to Compute Engine by selecting one of the following options:

    Select the tab for how you plan to use the samples on this page:

    gcloud

    1. After installing the Google Cloud CLI, initialize it by running the following command: 

      gcloud init

      If you're using an external identity provider (IdP), you must first sign in to the gcloud CLI with your federated identity.

    2. Set a default region and zone.

    REST

    To use the REST API samples on this page in a local development environment, you use the credentials you provide to the gcloud CLI.

      After installing the Google Cloud CLI, initialize it by running the following command: 

      gcloud init

      If you're using an external identity provider (IdP), you must first sign in to the gcloud CLI with your federated identity.

    For more information, see Authenticate for using REST in the Google Cloud authentication documentation.

OS adaptation script

To help you adapt images for Google Cloud, Google Cloud provides a script that completes the following steps:

  1. Creates an instance with the disk from Cloud Storage and runs the OS adaption tool on the instance's disk. This process installs the drivers that are required for your OS image to function on Google Cloud.
  2. Extracts the OS adaption report and copies the output files to your local machine. This report provides information about the results of the adaptation process, along with details about the detected OS. If the script doesn't run successfully, then the report provides errors, warnings, and action items to fix adaptation issues.
  3. Creates an image from the instance's disk. You can then use this image to create instances on Google Cloud.
  4. Deletes the instance and disk that were created during this process. This step cleans up unneeded resources from your project.

The following script adapts images for use in Google Cloud:

Full OS adaptation script

#!/usr/bin/env bash

# Sets variables

set -e
set -x
set -o pipefail

UNIQUE_ID=''
TPC_PREFIX=''
ZONE=''
PROJECT=''
INPUT_IMAGE=''
INPUT_DISK=''
IMAGE_NAME=''
ADAPTATION_INSTANCE_TYPE='c3-highcpu-4' # needs to support nested virtualization



usage() {
  echo ""
  echo "Usage:"
  echo "       $0 -g  -o -z -t  -p "
  echo ""
  echo ""
  echo "Parameters:"
  echo "       -i  OR -d  OR -g "
  echo "       -o "
  echo "       -z "
  echo "       -t "
  echo "       -p "
  echo "       -h (help)"
  echo ""
}

while getopts ":i:d:g:o:z:t:p:h" opt; do
  case "${opt}" in
    i)
      INPUT_IMAGE="${OPTARG}"
      ;;
    d)
      INPUT_DISK="${OPTARG}"
      ;;
    g)
      INPUT_GCS_OBJECT="${OPTARG}"
      ;;
    o)
      IMAGE_NAME="${OPTARG}"
      ;;
    z)
      ZONE="${OPTARG}"
      ;;
    t)
      TPC_PREFIX="${OPTARG}"
      ;;
    p)
      PROJECT="${OPTARG}"
      ;;
    h)
      echo "$0 help:"
      usage
      exit 0
      ;;
    *)
      echo "Unknown arg -${opt} ${OPTARG}"
      usage
      exit 1
      ;;
  esac
done
shift $((OPTIND-1))

# Defines errors

if [[ -n "${INPUT_IMAGE}" && -n "${INPUT_DISK}" ]] ||
   [[ -n "${INPUT_IMAGE}" && -n "${INPUT_GCS_OBJECT}" ]] ||
   [[ -n "${INPUT_DISK}" && -n "${INPUT_GCS_OBJECT}" ]]; then
  echo "Only one of -i, -d, or -g must be specified"
  usage
  exit 1
fi

if [[ -z "${IMAGE_NAME}" ]]; then
  echo "The name of the image must be specified"
  usage
  exit 1
fi

if [[ -z "${ZONE}" ]]; then
  echo "Zone must be specified"
  usage
  exit 1
fi

if [[ -z "${TPC_PREFIX}" ]]; then
  echo "TPC prefix must be specified"
  usage
  exit 1
fi

if [[ -z "${PROJECT}" ]]; then
  echo "Project must be specified"
  usage
  exit 1
fi

# Creates disk

OS_ADAPTATION_IMAGE="projects/${TPC_PREFIX}:v2v-community/global/images/family/v2v-tpc-stable"

if [[ -z "${UNIQUE_ID}" ]]; then
  UNIQUE_ID=$(head /dev/urandom | tr -dc a-z0-9 | head -c4)
fi

if [[ -n "${INPUT_IMAGE}" ]]; then
  DISK_ID="${IMAGE_NAME}-disk-${UNIQUE_ID}"
  echo "Creating disk ${DISK_ID} from image ${INPUT_IMAGE}..."
  gcloud compute disks create "${DISK_ID}" --zone="${ZONE}" --image "${INPUT_IMAGE}"
elif [[ -n "${INPUT_GCS_OBJECT}" ]]; then
  DISK_ID="${IMAGE_NAME}-disk-${UNIQUE_ID}"

  echo "Creating disk ${DISK_ID} from GCS object ${INPUT_GCS_OBJECT}..."
  CURL_OUTPUT=$(curl \
    -H "Authorization: Bearer $(gcloud auth print-access-token)" \
    -H 'Content-Type: application/json' \
    -X POST  \
    "https://compute.googleapis.com/compute/v1/projects/${TPC_PREFIX}:${PROJECT}/zones/${ZONE}/disks" \
    -d '{"name":"'"${DISK_ID}"'", "sourceStorageObject":"'"${INPUT_GCS_OBJECT}"'"}')
  OPERATION_ID=$(jq -r '.name' <<< "${CURL_OUTPUT}")
  while ! gcloud compute operations describe --zone "${ZONE}" --format=json \
    "${OPERATION_ID}" | jq -e '.status == "DONE"' >/dev/null; do
    sleep 5
  done
elif [[ -n "${INPUT_DISK}" ]]; then
  DISK_ID="${INPUT_DISK}"
fi

QUERY_PATH='v2v/report'
DEVICE_NAME="dev-${UNIQUE_ID}"
OUTPUT_DIR='/var/v2v'
STARTUP_SCRIPT="docker run --privileged -v /dev:/dev -v ${OUTPUT_DIR}:${OUTPUT_DIR} v2v-tpc -vv --log file=${OUTPUT_DIR}/output.log,level=trace --report file=${OUTPUT_DIR}/report.json --report file=http://metadata.google.internal/computeMetadata/v1/instance/guest-attributes/${QUERY_PATH},headers=Metadata-Flavor=Google --license=byol --in-place --drive file=/dev/disk/by-id/google-${DEVICE_NAME},format=raw"

# Creates instance and runs adaptation tool on disk

echo "Adapting disk ${DISK_ID}"

INSTANCE="os-adaptation-instance-${UNIQUE_ID}"
gcloud compute instances create "${INSTANCE}" \
  --zone="${ZONE}" \
  --machine-type="${ADAPTATION_INSTANCE_TYPE}" \
  --enable-nested-virtualization \
  --metadata="^@@@^enable-guest-attributes=TRUE@@@startup-script=${STARTUP_SCRIPT}" \
  --create-disk="auto-delete=yes,boot=yes,image=${OS_ADAPTATION_IMAGE}" \
  --disk="name=${DISK_ID},auto-delete=no,device-name=${DEVICE_NAME}"


# Retrieves report

while [[ -z "${REPORT}" ]]; do
  sleep 5
  set +e
  REPORT=$(gcloud compute instances get-guest-attributes "${INSTANCE}" \
    --zone="${ZONE}" \
    --query-path="${QUERY_PATH}" \
    --format='value(value)') 2>/dev/null
  echo -n '.'
  set -e
done
echo "Operating System adaptation report:"
jq . <<< "${REPORT}"

gcloud compute scp --recurse "${INSTANCE}:${OUTPUT_DIR}" "$(pwd)"
echo "Output file: $(pwd)/v2v/output.log, Report file: $(pwd)/v2v/report.json"

echo "Deleting ${INSTANCE}..."
gcloud compute instances delete "${INSTANCE}" --zone="${ZONE}" --quiet

if jq --exit-status '.status != "SUCCESS"' <<< "${REPORT}" >/dev/null; then
  echo "Operating System adaptation failed"
  exit 1
fi

# Creates image

echo "Operating System adaptation successful, creating image..."

IMAGE="image-${UNIQUE_ID}"
gcloud compute images create "${IMAGE}" \
  --source-disk-zone="${ZONE}" \
  --guest-os-features="$(jq --raw-output '.guest_os_features | join(",")' <<< "${REPORT}")" \
  --source-disk="${DISK_ID}"

if [[ -z "${INPUT_DISK}" ]]; then
  gcloud compute disks delete "${DISK_ID}" --zone="${ZONE}" --quiet
fi

echo "Adapted image: ${IMAGE}"

Run the OS adaptation script

To adapt your OS image for Google Cloud, do the following:

  1. Create a shell script.
  2. Copy the OS adaptation script and paste it into your shell script file.

  3. Run your shell script with the following flags:

    SCRIPT_NAME \
 -g 'INPUT_GCS_OBJECT' \
 -o 'IMAGE_NAME' \
 -z 'ZONE' \
 -p 'PROJECT' \
 -t 'TPC_PREFIX'

    Replace the following:

    • SCRIPT_NAME: the name of the OS adaptation script file
    • INPUT_GCS_OBJECT: the path to the disk in your Cloud Storage bucket
    • IMAGE_NAME: the name for the adapted image
    • ZONE: the zone for the adapted image
    • PROJECT: your Google Cloud project ID
    • TPC_PREFIX: a prefix used for your resource names

After the script runs successfully, it prints the following message:

Adapted image: IMAGE_NAME

What's next

Create an instance from your image.