Agentic AI use case: Orchestrate access to disparate enterprise systems

This document provides a high-level architecture for an agentic AI system to orchestrate interactions with disparate enterprise systems. An orchestrator agent automates complex processes and unifies access to multiple enterprise systems through modern, conversational interfaces. These systems might be commercial third-party applications or proprietary internal systems. This design approach eliminates point-to-point system integrations, so that your operators can avoid constant context-switching (swivel-chair processing).

The intended audience for this document includes architects, developers, and administrators who build and manage agentic AI applications in the cloud. The document assumes that you have a foundational understanding of agentic AI.

Architecture

The following diagram shows a multi-layer architecture where an orchestrator agent uses AI to route client interactions from multiple channels to appropriate backend systems.

Architecture for an agentic AI application that orchestrates access to disparate enterprise systems. Architecture for an agentic AI application that orchestrates access to disparate enterprise systems.

The components in the architecture are organized in the following layers:

Layer Components
Interaction channels

In the example architecture, clients interact with the agentic AI system through the following multiple channels:

  • A custom frontend, such as a web application that communicates with the agent over a secure REST API and supports human-in-the-loop processes.
  • An ad hoc chat interface on Gemini Enterprise that invokes the agent as a secure tool and lets users interact with complex backend systems by using natural language.
  • An external application that initiates system-to-system automation workflows by publishing business events to a Pub/Sub topic.
Agentic core

The core component of the architecture is an orchestrator agent that's built using Agent Development Kit (ADK) and deployed on Cloud Run. The orchestrator manages a multi-agent system. This serverless platform provides scalability and access control using Identity and Access Management (IAM) for the agent's REST API.

To maintain state across multi-step tasks, the agent uses built-in ADK support to persist state data in Vertex AI Session Service or Cloud Storage.

Standardized integration with backend systems

For communication between the agent and the backend systems, the architecture uses Model Context Protocol (MCP) servers that are deployed on Cloud Run.

  • MCP Toolbox for Databases provides a secure, out-of-the-box set of tools for access to databases like Cloud SQL.
  • Custom MCP servers provide access to the backend applications.

Each MCP server exposes the API of a specific backend system as a standardized set of tools. The MCP servers form an anti-corruption layer that isolates the agent from the backend systems. This approach helps avoid tight coupling of the agent with the backend systems. It simplifies the agent's logic and lets you modernize the backend systems independently of the agent.

Backend systems

In the example architecture, the AI agent orchestrates access to the following backend systems:

  • An operational database in Cloud SQL.
  • An application that's deployed on Compute Engine VMs.
  • An application that runs outside Google Cloud.

Products used

This example architecture uses the following Google Cloud products:

  • Vertex AI: An ML platform that lets you train and deploy ML models and AI applications, and customize LLMs for use in AI-powered applications.
  • Cloud Run: A serverless compute platform that lets you run containers directly on top of Google's scalable infrastructure.
  • Pub/Sub: An asynchronous and scalable messaging service that decouples services that produce messages from services that process those messages.
  • Gemini Enterprise: A fully-managed secure platform to deploy and manage AI agents within an enterprise.
  • Gemini: A family of multimodal AI models developed by Google.
  • Cloud Storage: A low-cost, no-limit object store for diverse data types. Data can be accessed from within and outside Google Cloud, and it's replicated across locations for redundancy.
  • Cloud SQL: A fully managed relational database service that helps you provision, operate, and manage your MySQL, PostgreSQL, and SQL Server databases on Google Cloud.

Use cases

This architecture is useful for the following use cases:

  • Automate swivel-chair processing between disparate external and internal applications.
  • Create a unified interface for multiple legacy systems without a large-scale migration.
  • Add intelligent, conversational capabilities to existing business processes.
  • Incrementally modernize your applications with event-driven functionality.

The architecture provides the following benefits:

  • Productivity: Automate repetitive tasks to reduce human error, so your teams can focus on value-added tasks.
  • Scalability: Use an event-driven approach to develop, deploy, and scale services independently.
  • Enhanced user experience: Build modern, flexible interfaces to integrate complex business processes.
  • Future-proof integration: Standardize integration with backend systems by using MCP servers, so you can replace and upgrade the backend systems independently.

Design considerations

To implement this architecture for production, consider the following recommendations:

  • Security: For the Cloud Run services, use dedicated IAM service accounts with permissions based on the principle of least privilege. Secure the agent's API by configuring Cloud Run ingress controls to restrict access to authenticated callers.
  • Observability: In distributed systems, logging and tracing are critical for troubleshooting. To get visibility into the entire agentic workflow, instrument your services to write structured logs to Cloud Logging and emit traces to Cloud Trace.
  • Performance: Cloud Run scales automatically based on demand and it scales to zero when there's no load. For latency-sensitive applications, you can minimize cold starts by configuring a minimum number of instances.
  • Deployment automation: Automate the provisioning and management of resources by using an infrastructure-as-code (IaC) tool like Terraform. IaC helps ensure repeatable, auditable deployments across development, staging, and production.
  • Governance: Streamline the design and deployment of your application's infrastructure by using Application Design Center. You can use App Design Center to define templates that incorporate governance rules and best practices for your organization.

What's next

Contributors

Author: Casey West | Architecture Advocate, Google Cloud

Other contributors: