This tutorial walks you through the process of testing the performance of your smart compose model using the Agent Assist console simulator. If you prefer, you can also perform all of the following actions by calling the API directly. You can use the console to train a model and test its performance, but be aware that all runtime operations must be carried out by calling the API directly.
Before you begin
- If you are using the publicly-available conversation dataset, review the documentation that guides you through using these resources.
- If you are using your own data, make sure that you have created one or more conversation datasets. At least one conversation dataset is required for model training.
- Create and deploy a smart compose model and manage its allowlist.
- Create a conversation profile that includes your smart compose model. You must create a conversation profile before you can use the simulator.
Test your smart compose model using the simulator
Navigate to the Agent Assist console and click the Conversation profiles menu option on the left side of the page:
The conversation profiles page displays a list of your conversation profiles. To use the simulator, click the three vertical dots to the far right of the conversation profile that you want to test and click Use simulator. The simulator page appears:
In the simulator, you can test your conversation profile's performance by inputting text and evaluating the returned suggestions. If your conversation profile is configured to enable more features than smart compose, you can choose which features to include in the simulation by clicking the Options button at the top right of the simulator page and making a selection. Make sure that smart compose is selected.
The following example tests a smart compose model's performance when a customer contacts a human agent to cancel an order. Smart compose is the only feature being tested in this example (all other options, such as smart reply, are unchecked under the Options menu).
The customer initiates the conversation by typing Hi. The human agent begins to respond by typing H. A smart compose suggestion appears in gray after this letter is typed:
The agent has the option to either hit the tab key to accept the smart compose suggestion or continue typing. Each time the agent either types a new character or hits tab to accept a suggestion, a new smart compose suggestion will appear. Suggestions are automatically generated until the sentence is completed. In this example, the agent hits tab to select the suggestion of How can I help, which generates another suggestion to add on to the sentence:
The agent again hits tab to accept the you today? suggestion and sends the response to the customer. The customer replies with I want to cancel my order. The agent begins to type a response asking the customer for the order number. Typing D brings up the suggestion Don't have the option to cancel your order, which is not the message that the agent wants to send:
Instead of accepting the suggestion, the agent continues to type. Do brings up the same suggestion as D, so the agent enters a space to continue their sentence. The space brings up the suggestion Do you know the order, which the agent hits tab to accept. The agent accepts the next smart compose suggestion of number? to complete the sentence: Do you know the order number?:
The customer responds with their order number. The agent then wants to ask them to wait while they look up the order number. O brings up the suggestion One moment which the agent accepts. The next suggestion is while I pull which they also accept, then up your order. which they also accept. The agent sends the completed sentence to the customer:
Smart compose will make a new suggestion each time the human agent types a character in the response box until the agent concludes their interaction with the customer and ends the conversation.
(Optional). Smart compose can be used in conjunction with smart reply. You must create a conversation profile that configures both smart compose and smart reply models to test both features at once using the simulator. If you have both features checked under the Options drop-down menu, the human agent will see bubbles containing smart reply text suggestions below the response box:
The agent can select a smart reply suggestion to copy it to the response box. If they choose to add another sentence or otherwise add characters to the suggestion, smart compose will make a suggestion to complete the agent's sentence.