BidiProcessOrderResponse

The top-level message returned from FoodOrderingService.BidiProcessOrder method.

JSON representation 
{
  "diagnosticInfo": {
    object (DiagnosticInfo)
  },

  // The following is a list of mutually exclusive fields. At most one of the
  // fields will be set in a response:
  "agentAudio": {
    object (AgentAudio)
  },
  "speechRecognition": {
    object (SpeechRecognition)
  },
  "legacyOrderStateResponse": {
    object
  },
  "interruptionSignal": {
    object (InterruptionSignal)
  },
  "agentEvent": {
    object (AgentEvent)
  },
  "clientExecutionRequest": {
    object (ClientExecutionRequest)
  },
  "updatedOrderState": {
    object (UpdatedOrderState)
  },
  "agentText": {
    object (AgentText)
  },
  "suggestedOptions": {
    object (SuggestedOptions)
  },
  "sentimentMeasurement": {
    object (SentimentMeasurement)
  },
  "turnCompleted": {
    object (TurnCompleted)
  },
  "endSession": {
    object (EndSession)
  },
  "recommendedOrders": {
    object (RecommendedOrders)
  }
  // End of mutually exclusive fields.
}
Fields
diagnosticInfo

object (DiagnosticInfo)

Diagnostic information containing execution timing, VAD speech boundaries, and latency metrics for the completed turn. Populated upon turn completion.
The response from the agent. The following is a list of mutually exclusive fields. At most one of the fields will be set in a response:
agentAudio

object (AgentAudio)

The audio response from the agent.
speechRecognition

object (SpeechRecognition)

The speech recognition result from the agent.
legacyOrderStateResponse

object (Struct format)

The legacy order state response for a session.
interruptionSignal

object (InterruptionSignal)

Interruption signal detected from the audio input.
agentEvent

object (AgentEvent)

Event from the agent.
clientExecutionRequest

object (ClientExecutionRequest)

The request forward to the client to execute. This is used for client side execution, such as adding a code to the order.
updatedOrderState

object (UpdatedOrderState)

Indicates an update to the order state. Includes the complete state of the order.
agentText

object (AgentText)

The text response from the agent.
suggestedOptions

object (SuggestedOptions)

Options to suggest to the customer.
sentimentMeasurement

object (SentimentMeasurement)

The realtime sentiment score measured for this turn.
turnCompleted

object (TurnCompleted)

Turn Completed message.
endSession

object (EndSession)

Indicates that the session is ending, and that this will be the last turn.

The agent will finish sending any pending responses for the current turn, and then send a final turnCompleted message. The agent will not start any new turns. For example, a 2 turn session will look like:

  • Turn 1
  • customer input 1
  • ... agent response and order state messages
  • turnCompleted
  • Turn 2
  • customer input 2
  • ... agent response and order state messages
  • endSession
  • ... pending agent response and order state messages
  • turnCompleted
recommendedOrders

object (RecommendedOrders)

The recommended orders from the agent.
End of mutually exclusive fields.

AgentAudio

The audio response from the agent.

JSON representation 
{
  "agentAudio": string
}
Fields
agentAudio

string (bytes format)

The audio data from the agent.

A base64-encoded string.

SpeechRecognition

The speech recognition result from the agent.

JSON representation 
{
  "transcript": string,
  "languageCode": string,
  "isFinal": boolean
}
Fields
transcript

string

Transcribed text from the audio input in the detected language.
languageCode

string

The language code of the transcript. The language code should be in BCP-47 format, such as "en-US".
isFinal

boolean

Indicates whether this is the final transcript for the turn.

InterruptionSignal

This type has no fields.

Indicates the agent's audio response has been interrupted (e.g. by end user barge-in). Client should stop playback immediately.

AgentEvent

Special event from the agent that needs client to take action.

JSON representation 
{

  // The following is a list of mutually exclusive fields. At most one of the
  // fields will be set in a response:
  "restartOrder": {
    object (RestartOrder)
  }
  // End of mutually exclusive fields.
}
Fields
The event from the agent. The following is a list of mutually exclusive fields. At most one of the fields will be set in a response:
restartOrder

object (RestartOrder)

Customer requests to restart the order.
End of mutually exclusive fields.

RestartOrder

Customer requests to start over the order.

JSON representation 
{
  "confirmationStep": boolean
}
Fields
confirmationStep

boolean

The agent is asking the customer to confirm the restarting request.

ClientExecutionRequest

A request to the for client side to execute a command.

JSON representation 
{
  "executionId": string,

  // The following is a list of mutually exclusive fields. At most one of the
  // fields will be set in a response:
  "finalizeOrderRequest": {
    object (FinalizeOrderRequest)
  },
  "submitOrderRequest": {
    object (SubmitOrderRequest)
  },
  "orderPricesRequest": {
    object (OrderPricesRequest)
  },
  "validateDeliveryAddressRequest": {
    object (ValidateDeliveryAddressRequest)
  },
  "orderStatusRequest": {
    object (OrderStatusRequest)
  }
  // End of mutually exclusive fields.
}
Fields
executionId

string

Required. A unique ID for this execution request.
The specific request to be executed. The following is a list of mutually exclusive fields. At most one of the fields will be set in a response:
finalizeOrderRequest

object (FinalizeOrderRequest)

A request to finalize the order.
submitOrderRequest

object (SubmitOrderRequest)

A request to submit the order.
orderPricesRequest

object (OrderPricesRequest)

A request to get the order prices.
validateDeliveryAddressRequest

object (ValidateDeliveryAddressRequest)

A request to validate the delivery address.
orderStatusRequest

object (OrderStatusRequest)

A request to get the order status.
End of mutually exclusive fields.

FinalizeOrderRequest

This type has no fields.

A request to finalize the order. This is typically sent by the agent when the customer indicates they are done ordering and want to pay.

SubmitOrderRequest

A request to submit the order, with all the required payment information collected from the customer.

JSON representation 
{
  "paymentInfo": {
    object (PaymentInfo)
  },
  "loyaltyInfo": {
    object (LoyaltyInfo)
  },
  "tipAmount": {
    object (Money)
  }
}
Fields
paymentInfo

object (PaymentInfo)

Required. The payment information for the order.
loyaltyInfo

object (LoyaltyInfo)

Optional. The loyalty consumed for the order.
tipAmount

object (Money)

Optional. The tip amount for the order.

OrderPricesRequest

A request to get the order prices.

JSON representation 
{
  "order": {
    object (Order)
  }
}
Fields
order

object (Order)

Optional. The order to get the prices for. If not specified, we can assume the request is for the existing order/cart in the session.

ValidateDeliveryAddressRequest

A request to validate a delivery address collected during the session.

JSON representation 
{
  "deliveryAddress": {
    object (DeliveryAddress)
  }
}
Fields
deliveryAddress

object (DeliveryAddress)

Required. The delivery address to validate.

OrderStatusRequest

This type has no fields.

A request to get the order status.

UpdatedOrderState

Event returned by the agent representing an update to the order state.

JSON representation 
{
  "order": {
    object (Order)
  },
  "changed": boolean,
  "status": enum (OrderStateUpdateStatus),
  "detailedMessage": string
}
Fields
order

object (Order)

The updated state of the Order associated with this session.
changed

boolean

Output only. Whether the order state has updates.
status

enum (OrderStateUpdateStatus)

Output only. Status of the order state update.
detailedMessage

string

Output only. Detailed message if the update failed.

OrderStateUpdateStatus

Status of the order state update.

Enums
ORDER_STATE_UPDATE_STATUS_UNSPECIFIED Unspecified status.
SUCCEEDED The update succeeded.
FAILED The update failed.

AgentText

The text response from the agent.

JSON representation 
{
  "text": string
}
Fields
text

string

The text response from the agent.

SentimentMeasurement

The realtime sentiment score measured for this turn.

JSON representation 
{
  "score": number,
  "magnitude": number
}
Fields
score

number

Optional. Sentiment score between -1.0 (negative sentiment) and 1.0 (positive sentiment).
magnitude

number

Optional. Magnitude of sentiment, regardless of score.

TurnCompleted

This type has no fields.

Indicates that the Food AI agent has detected the end of the current conversation turn and will provide no further output for this turn.

EndSession

Indicates the agent has terminated the session, due to either successful completion (e.g. user says "Good bye!" ) or an agent escalation.

JSON representation 
{
  "type": enum (EndType),
  "reason": string,
  "posResponse": string,
  "agentResponse": string,
  "storeSchedule": {
    object (Schedule)
  }
}
Fields
type

enum (EndType)

The type of end session event.
reason

string

The reason for ending the session.
posResponse

string

Optional. The POS response from checking in the order (optional). This is an opaque string payload representing the POS transaction status.
agentResponse

string

Optional. The final agent response message to be spoken/displayed to the user (optional).
storeSchedule

object (Schedule)

Optional. The store schedule. This is only set when the end type is STORE_CLOSED.

EndType

Type of end session event.

Enums
END_TYPE_UNSPECIFIED The end type is not specified.
DRIVE_OFF The customer drove off.
AGENT_ESCALATION The agent escalated the session to a human agent.
CREW_INTERJECTION A crew member interjected in the session.
ORDER_COMPLETED Order was successfully completed and no further action is needed.
PAYMENT_FLOW_SUCCESS The payment flow was successful and no further action is needed.
PAYMENT_FLOW_FAILURE The payment flow failed.
CLIENT_ESCALATION The client escalated the session to a human agent.
CALLER_HANGUP The customer hung up the call prematurely.
REQUESTED_CHECKOUT The customer requested to check out.
REQUESTED_ORDER_STATUS The customer asked about their order status.
STORE_CLOSED The store is closed.

RecommendedOrders

A list of recommended orders by the agent to the user.

JSON representation 
{
  "recommendedOrders": [
    {
      object (RecommendedOrder)
    }
  ]
}
Fields
recommendedOrders[]

object (RecommendedOrder)

The recommended orders to be displayed to the user.

RecommendedOrder

An order recommended by the agent to the user.

JSON representation 
{
  "order": {
    object (Order)
  }
}
Fields
order

object (Order)

The recommended order to be displayed to the user.

DiagnosticInfo

DiagnosticInfo provides strongly typed execution timing and latency metrics for a conversational turn.

JSON representation 
{
  "speechStartOffset": string,
  "speechEndOffset": string,
  "perceivedLatency": string
}
Fields
speechStartOffset

string (Duration format)

The duration relative to the start of the audio stream when user speech began.

A duration in seconds with up to nine fractional digits, ending with 's'. Example: "3.5s".
speechEndOffset

string (Duration format)

The duration relative to the start of the audio stream when user speech ended.

A duration in seconds with up to nine fractional digits, ending with 's'. Example: "3.5s".
perceivedLatency

string (Duration format)

Perceived latency for the completed turn, measured from acoustic end of speech to the first synthesized audio packet dispatched to the client. Includes the 200ms VAD silence confirmation window.

A duration in seconds with up to nine fractional digits, ending with 's'. Example: "3.5s".