Generative AI use case: Generate solutions for customer-support questions

Last reviewed 2026-11-11 UTC

This document provides a high-level architecture for an application that uses AI to generate solutions for support questions from customers.

The intended audience for this document includes architects, developers, and administrators who build and manage generative AI applications in the cloud. The document assumes that you have a foundational understanding of generative AI.

The Deployment section of this document provides code samples for AI-assisted customer support use cases.

Architecture

The following diagram shows an architecture for an AI-assisted support desk application in Google Cloud. The application receives questions from customers, retrieves relevant resources from a knowledge base, and then it generates solutions for the questions. The architecture is an implementation of the retrieval-augmented generation (RAG) approach.

Architecture for an application that uses AI to generate responses to support requests from customers. Architecture for an application that uses AI to generate responses to support requests from customers.

The application in this architecture consists of containerized services that are deployed in a Google Kubernetes Engine (GKE) cluster. The architecture shows the following flow:

  1. A customer submits a question to the support desk application.
  2. The support desk application passes the customer's question to the knowledge retriever service.
  3. The knowledge retriever service constructs and sends a prompt to Gemini Enterprise Agent Platform to retrieve resources that are relevant to the customer's question.
  4. The Gemini model on Agent Platform identifies relevant resources from a support knowledge base that's stored in Cloud Storage.
  5. Gemini returns the IDs of the relevant resources to the knowledge retriever service.
  6. The knowledge retriever service retrieves the relevant resources from Cloud Storage.
  7. The knowledge retriever service sends the customer's question and relevant resources to the solution generator service.
  8. The solution generator service sends the resources to Agent Platform, with a prompt to generate a detailed solution for the customer's question.
  9. Gemini generates a solution, such as step-by-step instructions or a video walkthrough.
  10. The solution generator service provides the solution to the customer through the support desk application.

Products used

This example architecture uses the following Google Cloud products:

  • Google Kubernetes Engine (GKE): A Kubernetes service that you can use to deploy and operate containerized applications at scale using Google's infrastructure.
  • Gemini Enterprise Agent Platform: A comprehensive platform that lets you build, scale, govern, and optimize enterprise‑grade AI agents.
  • Cloud Storage: A low-cost, no-limit object store for diverse data types. Data can be accessed from within and outside Google Cloud, and it's replicated across locations for redundancy.

Design considerations

This section describes design factors, best practices, and recommendations in alignment with the Google Cloud Well-Architected Framework. Consider this information when you use this reference architecture to develop a topology that meets your specific requirements for security, cost optimization, operational efficiency, and performance.

Security, privacy, and compliance

To design a topology in Google Cloud that meets your workload's security, privacy, and compliance requirements, consider the following recommendations:

  • To screen untrusted customer prompts and RAG responses for prompt injection, jailbreaks, and harmful content, use Model Armor.
  • To detect and redact personally identifiable information (PII) in support tickets and knowledge-base documents, integrate Sensitive Data Protection.

For security principles and recommendations that are specific to AI and ML workloads, see AI and ML perspective: Security in the Well-Architected Framework.

Cost optimization

This section provides guidance to help you optimize the cost of setting up and operating your customer-support topology in Google Cloud.

When support prompts repeatedly share system instructions and reference documents, enable context caching on Agent Platform. Reusing cached prefix tokens across requests reduces repeated input token costs and lowers response latency.

For cost optimization principles and recommendations that are specific to AI and ML workloads, see AI and ML perspective: Cost optimization in the Well-Architected Framework.

Operational efficiency

To design a Google Cloud topology that you can operate efficiently, consider the following recommendations:

For operational excellence principles and recommendations that are specific to AI and ML workloads, see AI and ML perspective: Operational excellence in the Well-Architected Framework.

Performance optimization

To design a topology in Google Cloud that meets the performance requirements of your workloads, consider the following recommendations:

  • To achieve low-latency semantic retrieval across large support knowledge bases, index document embeddings in Vector Search. To combine vector and keyword search with automatic embedding generation in a single collection, consider Agent Retrieval.
  • To maintain consistent response latency and avoid quota exhaustion during support traffic spikes, reserve capacity with Provisioned Throughput on Agent Platform.

For performance optimization principles and recommendations that are specific to AI and ML workloads, see AI and ML perspective: Performance optimization in the Well-Architected Framework.

Deployment

To experiment with AI-assisted customer support applications in Google Cloud, use the following code samples:

What's next

Contributors

Author: Kumar Dhanagopal | Cross-Product Solution Developer

Other contributors: