Data store search configuration

You can influence search results retrieved from Dialogflow CX data store tools by configuring boost and filter specifications. This lets you have more personalized and context-aware interactions when your agent uses a data store to find information.

Optionally, you can include dynamic expressions to fine-tune your results based on conversation context. For example, your agent has captured information indicating that the end-user owns a "mobile phone". You can configure the data store tool to boost documents related to mobile phones when answering a general query later in the conversation like "How do I check my voicemail?".

You can configure data store search results using the console, API, or Dialogflow CX Messenger integration.

Search condition inputs

You configure search results by using the boost specification (BoostSpec) and filter specification (FilterSpec) fields in a SearchConfig object. These configurations apply per data store within the tool, letting you control how each connected data store behaves.

You can configure search conditions in one of two ways: by using the console or by sending a direct API call.

There are important distinctions between the two:

• API call: You send BoostSpec and FilterSpec in a SearchConfig using a DetectIntent API call. You must provide a full SearchConfig object in the request. A SearchConfig sent by direct API call always overrides a SearchConfig sent using the console. Dynamic expressions and parameter references aren't supported.

• Console: The console uses your BoostSpec and FilterSpec configurations to construct a SearchConfig object that it sends with the search request. Optionally, you can include parameter references and dynamic expressions to tailor results to context data recorded from the conversation. You're only required to provide ConditionBoostSpec objects and a list of filter strings to construct FilterSpecs rather than a full SearchConfig object.

End-user information is provided as JSON in detectIntentRequest. There's no expected schema, so you can define the object properties.

Boost specifications (Boost specs)

Boost specs lets you change search result ranking by applying a boost value to specific documents. You can add multiple boost specifications to a single data store.

You enter each Boost Spec as a JSON string that represents a single ConditionBoostSpec object.

Key fields:

• condition: (String) An expression that specifies when the boost should apply. This uses the standard filter expression syntax. You can use Dialogflow CX expressions to make the results dynamic, such as $session.params.YOUR_PARAM_NAME or $request.end-user-metadata.YOUR_KEY.
• boost: (Number) A value between -1.0 and 1.0 that determines the strength of the boost.
  • A positive value promotes matching documents. A value of 1.0 gives a strong promotion.
  • A negative value demotes matching documents. A value of -1.0 gives a strong demotion.
  • A value of 0.0 applies no boost and isn't allowed.
• boostControlSpec: Provides more controls for a customized ranking than the basic combination of condition and boost. For more information about configuring this field, see the reference documentation.

Example console input:

If you configure your agent in the console, provide a list of ConditionBoostSpecs in the following format:

{
  "condition": "uri: ANY(\"http://www.example.com/docs/$session.params.doc_id\")",
  "boost": 0.5
}

Note: In this example, documents with a URI matching the value of the $session.params.doc_id session parameter are boosted with a strength of 0.5.

Example API input:

If you call the API directly, you must provide ConditionBoostSpecs in a full SearchConfig object. The following search configuration describes a boost spec:

"searchConfig": {
  "boostSpecs": [
    {
      "dataStores": [ "DATASTORE_ID" ],
      "spec": [
        {
          "conditionBoostSpecs": {
            "condition": "CONDITION",
            "boost": "1.0"
          }
        }
      ]
    }
  ]
}

Note: DATASTORE_ID is the full name of the data store: projects/your_project_id/locations/your_location/collections/your_collection_name/dataStores/your_datastore_name,
CONDITION must follow the Agent Search's filter expression syntax,
and the boost value must be between -1.0 and 1.0.

Filter Specifications (Filter specs)

Filter specifications restrict search results only to include documents that match the defined criteria. You can add multiple filter specifications to a single data store.

Each Filter Spec must be entered as a string expression. The string must conform to the standard filter expression syntax. You can use Dialogflow CX expressions within this string to make the results dynamic, such as $session.params.YOUR_PARAM_NAME or $request.end-user-metadata.YOUR_KEY.

Example Console Filter Spec string:

If you configure your agent using the console, you must provide a list of filter strings to form a FilterSpec object.

In this example, the filter only returns documents with numeric_field greater than or equal to the value of $session.params.min_value and where stock_availability is "IN_STOCK":

"numeric_field >= $session.params.min_value AND stock_availability: ANY(\"IN_STOCK\")"

Example API filter configuration:

If you call the API directly, you must provide filter strings in a full SearchConfig object:

"searchConfig": {
  "filterSpecs": [
    {
      "dataStores": [ "DATASTORE_ID" ],
      "filter": "CONDITION"
    }
  ]
}

Note: DATASTORE_ID is the full name of the data store: projects/your_project_id/locations/your_location/collections/your_collection_name/dataStores/your_datastore_name,
and CONDITION must follow the Agent Search's filter expression syntax.

Dialogflow CX dynamic expressions

Note: Dynamic expressions are available for data store tools only. They aren't available for data store handlers.

Both BoostSpec conditions and FilterSpec strings can incorporate Dialogflow CX expressions to make them dynamic. This lets you tailor search behavior based on context data retrieved from an ongoing conversation. Dynamic expressions aren't supported in direct API calls and can only be used if you're configuring using the console.

You can access conversation context data in two ways:

• Session parameters: The values collected during the conversation by using $session.params.YOUR_PARAMETER_ID.
• End-user metadata: The metadata about the end-user passed in the DetectIntentRequest by using $request.end-user-metadata.YOUR_KEY. To make this option available, verify that end_user_metadata is included in the QueryParameters of your DetectIntent calls. For more information, see endUserMetadata.

For more information about available system functions and expression syntax, see the conditions and system functions reference.

Search conditions applied at runtime

When your data store tool executes a search:

1. Evaluate the JSON strings that you provided for boost specifications. Each valid JSON string is converted into a ConditionBoostSpec object. These are then grouped into a BoostSpecs object for the specific data store connection, which is added to the overall SearchConfig.
2. Evaluate the strings that you provided for filter specifications as Dialogflow CX expressions. Each resulting filter string is used to create a FilterSpecs object for the data store, which is also added to the SearchConfig.
3. Include this dynamically constructed SearchConfig in the QueryParameters of the search request sent to the data store.

Configure search conditions

To configure search conditions, make sure that you've completed the following prerequisites:

• An existing Dialogflow CX agent.
• A data store tool configured for your agent with one or more data stores enabled.

Console configuration

1. Open the Conversational Agents console and choose a Google Cloud project.
2. Select an agent from the drop-down menu.
3. Navigate to the left-hand menu and click Tools. Select the data store tool that you want to configure.
4. In the tool editing page, navigate to the Data stores section. Click the Settings icon (⚙️) next to the data store you want to modify.
5. The Configure Data Store menu appears. Here you can add Boost Specs and Filter Specs to modify your search results.
   • For a Boost Spec, provide a JSON object that defines a ConditionBoostSpec. See Boost Specifications for details.
   • For a Filter Spec, provide a string that defines the filter criteria. See Filter Specifications for details.
6. After adding and configuring your specifications, click Confirm at the bottom of the side panel.
7. Click Save on the data store tool edit page to save the changes.

API configuration

You can provide search configuration data to Dialogflow CX when sending detect intent requests. You must provide this information in every detect intent request because it isn't persisted in the session.

Provide this information in the queryParams.searchConfig field in the Sessions.detectIntent method.

Go to the Session API reference

Select a protocol and version for the Session reference:

Protocol	V3	V3beta1
REST	Session resource	Session resource
RPC	Session interface	Session interface
C++	SessionsClient	Not available
C#	SessionsClient	Not available
Go	SessionsClient	Not available
Java	SessionsClient	SessionsClient
Node.js	SessionsClient	SessionsClient
PHP	Not available	Not available
Python	SessionsClient	SessionsClient
Ruby	Not available	Not available

Close

Dialogflow CX Messenger configuration

You can provide search configuration data to the Dialogflow CX Messenger integration. For more information, see the setContext method.

To apply a search spec or search config, you must add the following snippet to the Dialogflow CX Messenger code when embedding it into a website:

<script>
  document.addEventListener('df-messenger-loaded', () => {
    const dfMessenger = document.querySelector('df-messenger');
    const searchConfig = { ... }
    dfMessenger.setQueryParameters(searchConfig);
  });
</script>

For more information, see the setQueryParameters method.

Troubleshooting

The following solutions address some common issues encountered during configuration. Test your configurations thoroughly by simulating conversations that trigger different session parameters and end-user metadata values.

Invalid expressions

If a Boost Spec condition or a Filter Spec string contains an invalid Dialogflow CX expression (for example, incorrect syntax or reference to a nonexistent parameter), the expression compilation will fail. Errors related to expression compilation are typically returned in the DetectIntentResponse within the diagnostic_info field as SystemFunctionResults.

Invalid ConditionBoostSpec JSON

The Conversational Agents console performs some validation on the ConditionBoostSpec JSON string when saving it to check whether it's valid JSON and its structure can be mapped to a ConditionBoostSpec object. If the JSON is valid but it results in an invalid SearchConfig according to the underlying search service (for example, an invalid condition string after parameter substitution), the search service will return an error.

Runtime substitution errors

If a ConditionBoostSpec JSON string is valid and parsable but an error occurs during the runtime substitution of Dialogflow CX expressions within its fields (such as the condition string), these errors will be reported in diagnostic_info as SystemFunctionResults.

Review the compiled SearchConfig

The SearchConfig applied when the query is run is available in search_signals in the response. Reviewing the SearchConfig can provide insight into additional issues not outlined here.

What's next

• For more information about the structure of the SearchConfig and its components, see the search_config documentation.
• For more information about expression syntax, see the Dialogflow CX conditions and system functions reference.
• For more information about filter expression syntax for search, see Filter and order results.