This document provides a high-level architecture for an application that uses AI to generate solutions for support questions from customers.
The intended audience for this document includes architects, developers, and administrators who build and manage generative AI applications in the cloud. The document assumes that you have a foundational understanding of generative AI.
The Deployment section of this document provides code samples for AI-assisted customer support use cases.
Architecture
The following diagram shows an architecture for an AI-assisted support desk application in Google Cloud. The application receives questions from customers, retrieves relevant resources from a knowledge base, and then it generates solutions for the questions. The architecture is an implementation of the retrieval-augmented generation (RAG) approach.
The application in this architecture consists of containerized services that are deployed in a Google Kubernetes Engine (GKE) cluster. The architecture shows the following flow:
- A customer submits a question to the support desk application.
- The support desk application passes the customer's question to the knowledge retriever service.
- The knowledge retriever service constructs and sends a prompt to Gemini Enterprise Agent Platform to retrieve resources that are relevant to the customer's question.
- The Gemini model on Agent Platform identifies relevant resources from a support knowledge base that's stored in Cloud Storage.
- Gemini returns the IDs of the relevant resources to the knowledge retriever service.
- The knowledge retriever service retrieves the relevant resources from Cloud Storage.
- The knowledge retriever service sends the customer's question and relevant resources to the solution generator service.
- The solution generator service sends the resources to Agent Platform, with a prompt to generate a detailed solution for the customer's question.
- Gemini generates a solution, such as step-by-step instructions or a video walkthrough.
- The solution generator service provides the solution to the customer through the support desk application.
Products used
This example architecture uses the following Google Cloud products:
- Google Kubernetes Engine (GKE): A Kubernetes service that you can use to deploy and operate containerized applications at scale using Google's infrastructure.
- Gemini Enterprise Agent Platform: A comprehensive platform that lets you build, scale, govern, and optimize enterprise‑grade AI agents.
- Cloud Storage: A low-cost, no-limit object store for diverse data types. Data can be accessed from within and outside Google Cloud, and it's replicated across locations for redundancy.
Design considerations
This section describes design factors, best practices, and recommendations in alignment with the Google Cloud Well-Architected Framework. Consider this information when you use this reference architecture to develop a topology that meets your specific requirements for security, cost optimization, operational efficiency, and performance.
Security, privacy, and compliance
To design a topology in Google Cloud that meets your workload's security, privacy, and compliance requirements, consider the following recommendations:
- To screen untrusted customer prompts and RAG responses for prompt injection, jailbreaks, and harmful content, use Model Armor.
- To detect and redact personally identifiable information (PII) in support tickets and knowledge-base documents, integrate Sensitive Data Protection.
For security principles and recommendations that are specific to AI and ML workloads, see AI and ML perspective: Security in the Well-Architected Framework.
Cost optimization
This section provides guidance to help you optimize the cost of setting up and operating your customer-support topology in Google Cloud.
When support prompts repeatedly share system instructions and reference documents, enable context caching on Agent Platform. Reusing cached prefix tokens across requests reduces repeated input token costs and lowers response latency.
For cost optimization principles and recommendations that are specific to AI and ML workloads, see AI and ML perspective: Cost optimization in the Well-Architected Framework.
Operational efficiency
To design a Google Cloud topology that you can operate efficiently, consider the following recommendations:
- To automate document chunking, embeddings, and grounded retrieval over Cloud Storage, use RAG Engine on Gemini Enterprise Agent Platform. For out-of-the-box enterprise search with citations over unstructured support articles, you can alternatively ground responses with Agent Search.
- To orchestrate multi-turn support workflows and tool calls, build your services with Agent Development Kit (ADK). To persist conversation state across turns, use Gemini Enterprise Agent Platform Sessions. To retain long-term customer context across support sessions, pair Agent Platform Sessions with Memory Bank.
- If your team doesn't need cluster-level GKE customization, you can deploy support agents on Agent Runtime on Gemini Enterprise Agent Platform. For stateless containerized HTTP services, you can also run workloads on Cloud Run to reduce infrastructure management overhead.
- Before you deploy updates to prompts, retrieval logic, or the model, measure groundedness and answer quality with model and agent evaluation on Agent Platform.
For operational excellence principles and recommendations that are specific to AI and ML workloads, see AI and ML perspective: Operational excellence in the Well-Architected Framework.
Performance optimization
To design a topology in Google Cloud that meets the performance requirements of your workloads, consider the following recommendations:
- To achieve low-latency semantic retrieval across large support knowledge bases, index document embeddings in Vector Search. To combine vector and keyword search with automatic embedding generation in a single collection, consider Agent Retrieval.
- To maintain consistent response latency and avoid quota exhaustion during support traffic spikes, reserve capacity with Provisioned Throughput on Agent Platform.
For performance optimization principles and recommendations that are specific to AI and ML workloads, see AI and ML perspective: Performance optimization in the Well-Architected Framework.
Deployment
To experiment with AI-assisted customer support applications in Google Cloud, use the following code samples:
- Build a customer support application that uses generative AI.
- Design chat prompts for customer service chatbots.
What's next
- Build a customer support agent using Gemini.
- Build AI agents for customer support use cases by using instructions, flow-based agents, and data store tools in Customer Experience Agent Studio.
- Explore more generative AI architecture guides.
- For an overview of architectural principles and recommendations that are specific to AI and ML workloads in Google Cloud, see the AI and ML perspective in the Well-Architected Framework.
- For more reference architectures, diagrams, and best practices, explore the Cloud Architecture Center.
Contributors
Author: Kumar Dhanagopal | Cross-Product Solution Developer
Other contributors:
- Amina Mansour | Tech Lead, Global Developer Relations & Strategic Content
- Megan O'Keefe | Developer Advocate
- Samantha He | Technical Writer
- Shir Meir Lador | Developer Relations Engineering Manager