Food Ordering AI Agent concepts

This document provides a high-level overview of the key concepts in the Food Ordering AI Agent API.

Agent configuration

The behavior of the Food Ordering AI Agent is influenced by the configuration of several API resources: Brand, Store, and Menu. These resources define the restaurant's identity, its physical locations, and the products it offers, providing the necessary context for the AI agent to handle orders.

Brand

A Brand is the top-level resource, representing a restaurant brand corresponding to one or more locations of that restaurant brand. It contains configuration which is shared across all locations of that restaurant. Brand can include configuration of many features of the agent persona, such as greeting behavior and voice characteristics. Many of those features can be overridden by values configured in the Store resource or even in per-session configuration (see Session lifecycle)

Store

A Store resource represents a single physical restaurant location belonging to a Brand. It defines configuration specific to that location, such as its time zone, status (e.g., ACTIVE, DISABLED), operating hours, and dayparts (e.g., periods like "Breakfast" or "Lunch" during which certain menu items are available).

A Menu resource defines all the products offered by a restaurant, including all possible choices and customizations for every sellable product. A Menu must be associated with a Store. Menu is designed to be flexible and accommodate various menu structures, from small lists of standalone items to complex trees of combination meals with nested modifiers.

Key components of a Menu include:

  • Items: Sellable top-level products like a-la-carte entrees, drinks, sides, or combination meals.
  • ModifierGroups: Collections of choices applicable to an Item or another Modifier, such as "Choose a Side" or "Add Toppings".
  • Modifiers: Individual options within a ModifierGroup, like "Fries", "Extra Cheese", or "Cola". Modifiers can adjust the item price and can contain nested ModifierGroups for further customization.
  • MenuCategories: Organizational units like "Appetizers" or "Drinks".

A Menu resource is identified by a name in the following format: projects/{project}/locations/{location}/menus/{menu}.

For more details on structuring menu data, see Integrating menu data.

Food ordering sessions

Food ordering sessions are at the core of Food Ordering AI Agent, enabling real-time, multimodal interactions between a customer and the AI agent. Each session represents a single food ordering conversation and is managed using the FoodOrderingService.BidiProcessOrder RPC method.

BidiProcessOrder RPC method

This is a bidirectional streaming RPC: the client application streams input to the agent, and the agent concurrently streams responses back to the client. This allows for low-latency, real-time interactions.

  • Client-to-Agent Stream: The client sends a stream of BidiProcessOrderRequest messages containing audio input (customer speech), text input, or event inputs (such as a client-side cart update performed by a customer using a tap interface, or a hardware-detected drive-off event from drive-thru restaurant hardware).
  • Agent-to-Client Stream: The agent returns a stream of BidiProcessOrderResponse messages containing audio output (synthesized agent speech), text output, transcripts of recognized speech, updates to the customer's order state, or other signals such as detected interruptions.

Session lifecycle

Every Session in Food Ordering AI Agent must begin with client-provided configuration specified using a BidiProcessOrderRequest message containing a Config. The Config must specify two fields:

  • store: The full resource name of the Store for which the order is being placed (e.g. projects/PROJECT/locations/LOCATION/brands/BRAND/stores/STORE). The session takes on configuration specified in the referenced Store resource and that store's parent Brand resource. Where configuration conflicts across the Brand and Store, the Store configuration takes precedence.
  • session: A unique session identifier in the format projects/PROJECT/locations/LOCATION/sessions/SESSION. The session_id is a client-generated ID that uniquely identifies one customer interaction or conversation.