In publishing, a publisher client sends a message to a Pub/Sub
topic. Here are some best practices for publishing messages to
Pub/Sub.

This document assumes that you are already familiar with the process of
publishing messages to a Pub/Sub topic.

If you're new to Pub/Sub, see one of the Quickstart guides
and learn how to run Pub/Sub using the
[console](https://docs.cloud.google.com/pubsub/docs/publish-receive-messages-console),
[gcloud CLI](https://docs.cloud.google.com/pubsub/docs/publish-receive-messages-gcloud), or the
[client libraries](https://docs.cloud.google.com/pubsub/docs/publish-receive-messages-client-library).

## Take action based on the response from the publish

When the high-level client library's publish call completes, it returns a
future object that contains the result of the operation. To avoid blocking
individual publish requests, handle the result asynchronously. You should decide
the best way to handle the failure for your use case. Some options include:

- Log the error and do nothing else (if your use case does not require successful publish of all messages).
- Retry the publish on a potentially transient failure like a `Deadline
  exceeded` error.
- Persist the message to a file or storage to retry its publish later, especially on an error that requires user intervention like `Not found` or `Permission denied`.
- Propagate the errors to the upstream service that sent you the message you tried to publish.

If you believe Pub/Sub is not delivering messages as expected to
your subscribers, verify that you are tracking the results of publishes
and that the publishes were successful.

## Attach a subscription or enable topic retention before you begin to publish

If you start publishing to a topic that does not have an attached subscriber,
the messages are not retained. These messages can't be delivered to
subsequently attached subscriptions. Hence, before you start publishing
messages, do one of the following:

- Attach a subscription to a topic. Choose one of the following methods:

  - Create a subscription and specify a topic during the process. Learn how
    to create a [pull subscription](https://docs.cloud.google.com/pubsub/docs/create-subscription),
    [push subscription](https://docs.cloud.google.com/pubsub/docs/create-push-subscription),
    [BigQuery subscription](https://docs.cloud.google.com/pubsub/docs/create-bigquery-subscription), or a
    [Cloud Storage subscription](https://docs.cloud.google.com/pubsub/docs/create-cloudstorage-subscription).

  - [Select a default subscription when you create a topic](https://docs.cloud.google.com/pubsub/docs/create-topic).

- Enable topic message retention.

  [Topic message retention](https://docs.cloud.google.com/pubsub/docs/replay-overview#configuring_message_retention)
  lets a subscription replay messages that are published before you created
  a subscription. If topic message retention is enabled, storage costs for
  the messages retained by the topic are billed to the project where the
  topic is located.

## Configure batch messaging

Within Pub/Sub, batch messaging refers to the process of
combining multiple messages into one batch which gets published in a single
publish request. If you use [client libraries](https://docs.cloud.google.com/pubsub/docs/reference/libraries)
to publish your messages, batching is enabled by default.
The batching (or grouping) of messages helps the publisher
to improve its efficiency and send messages at a higher throughput. Batching
reduces the cost to publish data. However, batching also creates latency for
individual messages because the publisher waits for the batch to be filled up
before publishing the batch.

Latency in Pub/Sub can be of two types:

- **End-to-end latency** is the time it takes for a message to be published by
  a publisher and delivered to the corresponding subscribers for processing.

- **Publish latency** is the amount of time it takes to publish a message.

When using batching, increasing both types of latencies is a trade-off for
improving efficiency and throughput.

You can batch messages in a client library based on **message request size** ,
**number of messages** , and **time**. When configuring batch settings, you can
find the right balance between cost, throughput, and latency to suit your use
case.

The default values for the batch messaging variables and the names of the
variables might differ across client libraries. You can specify one or all three
values in the client library. If any one of the values for the batch messaging
variables is met, the client library publishes the next batch of messages.

To configure batch messaging for a publisher client, see
[Batch messages in a publish request](https://docs.cloud.google.com/pubsub/docs/batch-messaging).

## Configure flow control for transient message spikes

If the publisher client has to process a large number of messages,
publish requests might start to accumulate in memory until messages fail to
publish with a `Deadline exceeded` error.

To address transient spikes in publishing messages, you can use flow control in
your publisher settings. Publisher-side flow control prevents the publisher
client resources from becoming overwhelmed with too many outstanding requests.
If the publisher client becomes constrained in terms of memory, CPU, or threads,
a large number of `Deadline exceeded` errors are generated.

To configure flow control in the client library, set appropriate values for the
**maximum outstanding messages** and the **maximum outstanding message bytes**
variables. These values balance message throughput and system capacity.

To check if your client library supports publisher flow control and to configure
it, see [Flow control](https://docs.cloud.google.com/pubsub/docs/flow-control-messages).

## Understand your network bandwidth and latency

Your publisher throughput is constrained by your network bandwidth and the
number of requests sent. If your bandwidth is good but your network latency is
high, you don't want to overwhelm the system with many small requests.
Publisher-side flow control can help with client-side network issues.

Your publisher throughput is also CPU and memory bound. More available machine
cores lets you set higher thread count for better publish throughput. To further
understand how to maximize streaming performance, see
[Testing Cloud Pub/Sub clients to maximize streaming performance](https://cloud.google.com/blog/products/data-analytics/testing-cloud-pubsub-clients-to-maximize-streaming-performance).

## Tweak the retry request variables for failed publishes

When a message is published by a publisher client, you might see publishing
failures. These failures are typically caused by client-side bottlenecks, such
as insufficient service CPUs, bad thread health, or network congestion. The
`publisher retry policy` determines the behavior in case of message delivery
failure. The retry policy defines the number of times Pub/Sub
tries to deliver the message and the length of time between each attempt.

For example, in the
[Java client library for Pub/Sub](https://docs.cloud.google.com/pubsub/docs/retry-requests#retry_a_message_request),
the publisher client contains the following values:

- **initialRetryDelay.** The initial delay that the publisher waits
  before retrying a publish operation. The default value is `100
  milliseconds`.

- **retryDelayMultiplier.** The multiplication factor used to calculate the
  delay between retries. The default value is `4`. This means that the delay
  between retries is up to `100 milliseconds * 4 = 400 milliseconds` for the second
  retry, and up to `400 milliseconds * 4 = 1600 milliseconds` for the third retry.

- **maxRetryDelay.** The maximum delay that the publisher waits before
  retrying a publish operation. The default value is `60 seconds`.

- **initialRpcTimeout.** The initial timeout that the publisher waits
  for the RPC call to complete. The default value is `5 seconds`.

- **rpcTimeoutMultiplier.** The multiplication factor used to calculate the
  RPC timeout. The default value is `4.0`. This means that the
  timeout for the RPC call is up to `5 seconds * 4 = 20 seconds` for the second
  retry, and up to `10 seconds * 4 = 40 seconds` for the third retry.

- **maxRpcTimeout.** The maximum timeout that the publisher waits for
  the RPC call to complete. The default value is `600 seconds`.

- **totalTimeout.** The total timeout for the publish operation. This
  includes the time spent waiting for the RPC call to complete and the time
  spent waiting between retries. The default value is `600 seconds`.

Only make adjustments to the specified values if you find the default retry
settings are not sufficient for your use case. For example, publishing a large
number of messages wouldn't require you to increase the
`initialRetryDelay` and `maxRetryDelay` values. However, you can adjust
flow control and batching in such circumstances. If you're publishing from a
flaky internet connection or a connection that is bandwidth constrained,
you can experiment with the values for the `initialRpcTimeout`, `maxRpcTimeout`,
and `rpcTimeoutMultiplier` variables. For
recommended values, see
[Publish operations fail with DEADLINE_EXCEEDED](https://docs.cloud.google.com/pubsub/docs/topic-troubleshooting#publish_operations_fail_with_deadline_exceeded).

## Use message storage policy to ensure data locality

Pub/Sub's topic message storage policy offers a way to ensure
that messages published to a topic are never persisted outside a set of
Google Cloud regions that you specify, independent of where the publish requests
originate.

Use the message storage policy to specify a list of Google Cloud regions
where Pub/Sub is allowed to store message data on disk. When a
message is published to a region not in this list, the request is forwarded to
the nearest allowed region for processing. The policy can be configured on a
topic or as an organizational policy for a project, project folder, or an entire
organization. When an organization policy is configured, individual topic policy
can be changed only in ways that don't violate the organization policy.

For example, a company that operates in Europe might use the message storage
policy to ensure that all of the data is stored in EU regions to comply with
local laws.

For more information, see
[Configure message storage policies](https://docs.cloud.google.com/pubsub/docs/resource-location-restriction).

## Best practices for ordered messaging in publishing

If you use message ordering, ensure the following:

- **Use locational endpoints** . The ordering of messages is preserved on the
  publish side and within a region. In other words, if you are publishing
  messages to multiple regions, only messages within the same region are
  delivered in a consistent order. If your messages are all published to the
  same region, but your subscribers are spread across regions, then the
  subscribers receive all messages in order. Use a
  [locational endpoint](https://docs.cloud.google.com/pubsub/docs/reference/service_apis_overview#pubsub_endpoints)
  to publish messages to the same region.

- **Configure a resume publishing function** . When a client library retries a
  request and the message has an ordering key, the client library repeatedly
  retries the request, independent of the retry settings. If a non-retryable
  error occurs, the client library doesn't publish the message and stops
  publishing other messages with the same ordering key. When
  you are ready to continue publishing on an ordering key that has a failed
  publish, call the
  [`resumePublish`](https://docs.cloud.google.com/pubsub/docs/retry-requests#retry_ordering) method.

## Summary of best practices

The following table summarizes the best practices recommended in this document:

| Topic | Task |
|---|---|
| [Configure message retention](https://docs.cloud.google.com/pubsub/docs/replay-overview#configuring_message_retention) | Attach a subscription before you publish or enable message retention. |
| [Batch messages in a publish request](https://docs.cloud.google.com/pubsub/docs/batch-messaging) | Batch or group messages to increase the efficiency of the publisher and send messages at a higher throughput. |
| [Flow control](https://docs.cloud.google.com/pubsub/docs/flow-control-messages) | Configure flow control in your publisher settings to handle transient traffic spikes. |
| [Testing Pub/Sub clients to maximize streaming performance](https://cloud.google.com/blog/products/data-analytics/testing-cloud-pubsub-clients-to-maximize-streaming-performance) | Scale publisher throughput with an increase in available machine cores and network bandwidth. |
| [Retry requests](https://docs.cloud.google.com/pubsub/docs/retry-requests) | Make adjustments to the specified values of the publisher retry policy only if you find the default settings are not sufficient for your use case. |
| [Configure message storage policies](https://docs.cloud.google.com/pubsub/docs/resource-location-restriction) | Use the message storage policy to store message data on disk only in specific locations. |
| [Use a locational endpoint when using ordering keys in publishing](https://docs.cloud.google.com/pubsub/docs/publisher#using-ordering-keys) | When using ordered messaging, use a locational endpoint and configure a resume publishing function for publishing failures. |

## What's next

- [Best practices to subscribe to a Pub/Sub topic](https://docs.cloud.google.com/pubsub/docs/subscribe-best-practices).

- [Best practices for using Pub/Sub metrics as a scaling signal](https://docs.cloud.google.com/pubsub/docs/metrics-autoscaling-best-practices).