When building your Pub/Sub system, payload unwrapping can help you connect to other systems that don't adhere to all system requirements of a standard Pub/Sub push endpoint implementation. Some potential use cases for payload unwrapping are as follows: - You don't want to write Pub/Sub-specific message parsing code for your HTTP push endpoints. - You prefer receiving Pub/Sub message metadata as HTTP headers instead of the metadata in the HTTP POST body. - You want to send Pub/Sub messages and exclude the Pub/Sub metadata, for example when sending data to a third-party API. ## How payload unwrapping works Payload unwrapping is a feature that strips Pub/Sub messages of all message metadata, except for the message data. By sending raw message data, subscribers can process the message without having to adhere to any system requirements of Pub/Sub. - With payload unwrapping, the message data is delivered directly as the HTTP body. - Without payload unwrapping, Pub/Sub delivers a JSON object that contains multiple message metadata fields and a message data field. In this case, the JSON must be parsed to retrieve the message data and then base64 decoded. **Write metadata** After enabling payload unwrapping, you can use the *write metadata* option which adds previously removed message metadata into the [request header](https://docs.cloud.google.com/pubsub/docs/payload-unwrapping.md.txt#set-header). - **Write metadata enabled.** Add message metadata back into the request header. Also delivers the raw, decoded message data. - **Write metadata disabled.** Only delivers the raw, decoded message data. Write metadata is exposed through the Pub/Sub, the Google Cloud CLI argument `--push-no-wrapper-write-metadata`, and the API property [`NoWrapper`](https://docs.cloud.google.com/pubsub/docs/reference/rest/v1/projects.subscriptions.md.txt#NoWrapper). By default, this value is null. ## Before you begin - Learn about Pub/Sub [subscriptions](https://docs.cloud.google.com/pubsub/docs/subscriber.md.txt) and [push subscriptions](https://docs.cloud.google.com/pubsub/docs/push.md.txt). Payload unwrapping can only be used with push subscriptions. - Learn how to configure a [push subscription](https://docs.cloud.google.com/pubsub/docs/create-push-subscription.md.txt). ## Example for wrapped and unwrapped messages The following examples illustrate the difference between sending a wrapped and unwrapped HTTP message. In these examples, the message data contains the string `{"status": "Hello there"}`. For this example, a subscription is created with the payload unwrapping feature enabled and publishes a message to `mytopic`. It uses an ordering key with a value of `some-key` and the media type is declared as `application/json`. ``` gcloud pubsub topics publish mytopic --message='{"status": "Hello there"}' --ordering-key="some-key" --attribute "Content-Type=application/json" ``` The following sections show the difference between a wrapped and unwrapped message. ### Wrapped message The following example shows a standard Pub/Sub wrapped message. In this case, payload unwrapping isn't enabled. | Publish | Push Endpoint Receives | |---|---| | ``` data="{"status": "Hello there"}" ordering_key="some-key" attributes= { {"Content-Type", "application/json"} } ``` | ``` Content-Length: 361 Content-Type: application/json User-Agent: CloudPubSub-Google Host: subscription-project.uc.r.appspot.com { "message": { "attributes": { "Content-Type": "application/json" }, "data": "eyJzdGF0dXMiOiAiSGVsbG8gdGhlcmUifQ==", // Base64 - {"status": "Hello there"} "messageId": "2070443601311540", "message_id": "2070443601311540", "publishTime": "2021-02-26T19:13:55.749Z", "publish_time": "2021-02-26T19:13:55.749Z" }, "subscription": "projects/myproject/..." } ``` | ### Unwrapped message with write metadata disabled The following example shows an unwrapped message with the write metadata option disabled. In this case, the `x-goog-pubsub-*` headers and [message attributes](https://docs.cloud.google.com/pubsub/docs/reference/rest/v1/PubsubMessage.md.txt) aren't included. | Publish | Push Endpoint Receives | |---|---| | ``` data="{"status": "Hello there"}" ordering_key="some-key" attributes= { {"Content-Type", "application/json"} } ``` | ``` Content-Length: 25 User-Agent: CloudPubSub-Google Host: subscription-project.uc.r.appspot.com {"status": "Hello there"} ``` | ### Unwrapped message with write metadata enabled The following example shows an unwrapped message with the write metadata option enabled. In this case, the `x-goog-pubsub-*` headers and [message attributes](https://docs.cloud.google.com/pubsub/docs/reference/rest/v1/PubsubMessage.md.txt) are included. | Publish | Push Endpoint Receives | |---|---| | ``` data="{"status": "Hello there"}" ordering_key="some-key" attributes= { {"Content-Type", "application/json"} } ``` | ``` x-goog-pubsub-subscription-name: "projects/myproject/..." x-goog-pubsub-message-id: "2070443601311540" x-goog-pubsub-publish-time: "2021-02-26T19:13:55.749Z" x-goog-pubsub-ordering-key: "some-key" Content-Type: application/json Content-Length: 12 User-Agent: CloudPubSub-Google Host: subscription-project.uc.r.appspot.com {"status": "Hello there"} ``` | ## Configure payload unwrapping You can enable payload unwrapping push delivery for a subscription using the Google Cloud console **Subscription Details** page, the Google Cloud CLI, or the Client Libraries. ### Console 1. In the Google Cloud console, go to the **Subscriptions** page. [Open Pub/Sub subscriptions](https://console.cloud.google.com/cloudpubsub/subscription) 2. Click **Create subscription**. 3. In the **Subscription ID** field, enter a name. For information on how to name a subscription, see [Guidelines to name a topic or a subscription](https://docs.cloud.google.com/pubsub/docs/pubsub-basics.md.txt#resource_names). 4. Select a topic from the drop-down menu. The subscription receives messages from the topic. 5. For **Delivery type** , select **Push**. 6. To enable payload unwrapping, select **Enable payload unwrapping**. 7. (Optional) To preserve metadata of messages in the request header, select **Write metadata** . You must enable this option to set a [Content-Type header](https://docs.cloud.google.com/pubsub/docs/payload-unwrapping.md.txt#set-header) for your messages. 8. Specify an endpoint URL. 9. Retain all other default values. 10. Click **Create**. ### gcloud To configure a subscription with payload unwrapping that includes standard HTTP headers, run the following [`gcloud pubsub subscriptions create`](https://docs.cloud.google.com/sdk/gcloud/reference/pubsub/subscriptions/create) command: ``` gcloud pubsub subscriptions create SUBSCRIPTION \ --topic TOPIC \ --push-endpoint=PUSH_ENDPOINT \ --push-no-wrapper ``` Replace the following: - `SUBSCRIPTION`: the name or ID of your push subscription. - `TOPIC`: the ID of the topic. - `PUSH_ENDPOINT`: the URL to use as the endpoint for this subscription. For example, `https://myproject.appspot.com/myhandler` - `--push-no-wrapper`: delivers the message data directly as the HTTP body. To configure a subscription with payload unwrapping and control the use of `x-goog-pubsub-*` headers, run the following command: ``` gcloud pubsub subscriptions create SUBSCRIPTION \ --topic TOPIC \ --push-endpoint=PUSH_ENDPOINT \ --push-no-wrapper \ --push-no-wrapper-write-metadata ``` - `--push-no-wrapper-write-metadata`: When true, writes the Pub/Sub message metadata to `x-goog-pubsub-:` headers of the HTTP request. Writes the Pub/Sub message attributes to `:` headers of the HTTP request. ### Python Before trying this sample, follow the Python setup instructions in [Quickstart: Using Client Libraries](https://docs.cloud.google.com/pubsub/docs/create-topic-client-libraries). For more information, see the [Pub/Sub Python API reference documentation](https://docs.cloud.google.com/python/docs/reference/pubsub/latest.md.txt). ```python from google.cloud import pubsub_v1 # TODO(developer) # project_id = "your-project-id" # topic_id = "your-topic-id" # subscription_id = "your-subscription-id" # endpoint = "https://my-test-project.appspot.com/push" publisher = pubsub_v1.PublisherClient() subscriber = pubsub_v1.SubscriberClient() topic_path = publisher.topic_path(project_id, topic_id) subscription_path = subscriber.subscription_path(project_id, subscription_id) no_wrapper = pubsub_v1.types.PushConfig.NoWrapper(write_metadata=True) push_config = pubsub_v1.types.PushConfig( push_endpoint=endpoint, no_wrapper=no_wrapper ) # Wrap the subscriber in a 'with' block to automatically call close() to # close the underlying gRPC channel when done. with subscriber: subscription = subscriber.create_subscription( request={ "name": subscription_path, "topic": topic_path, "push_config": push_config, } ) print(f"Push no wrapper subscription created: {subscription}.") print(f"Endpoint for subscription is: {endpoint}") print(f"No wrapper configuration for subscription is: {no_wrapper}") ``` ### Java Before trying this sample, follow the Java setup instructions in [Quickstart: Using Client Libraries](https://docs.cloud.google.com/pubsub/docs/create-topic-client-libraries). For more information, see the [Pub/Sub Java API reference documentation](https://cloud.google.com/java/docs/reference/google-cloud-pubsub/latest/overview). ```java /* * Copyright 2016 Google LLC * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * * http://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. * See the License for the specific language governing permissions and * limitations under the License. */ package pubsub; import com.google.cloud.pubsub.v1.SubscriptionAdminClient; import com.google.pubsub.v1.PushConfig; import com.google.pubsub.v1.PushConfig.NoWrapper; import com.google.pubsub.v1.Subscription; import com.google.pubsub.v1.SubscriptionName; import com.google.pubsub.v1.TopicName; import java.io.IOException; public class CreateUnwrappedPushSubscriptionExample { public static void main(String... args) throws Exception { // TODO(developer): Replace these variables before running the sample. String projectId = "your-project-id"; String subscriptionId = "your-subscription-id"; String topicId = "your-topic-id"; String pushEndpoint = "https://my-test-project.appspot.com/push"; createPushSubscriptionExample(projectId, subscriptionId, topicId, pushEndpoint); } public static void createPushSubscriptionExample( String projectId, String subscriptionId, String topicId, String pushEndpoint) throws IOException { try (SubscriptionAdminClient subscriptionAdminClient = SubscriptionAdminClient.create()) { TopicName topicName = TopicName.of(projectId, topicId); SubscriptionName subscriptionName = SubscriptionName.of(projectId, subscriptionId); NoWrapper noWrapper = NoWrapper.newBuilder() // Determines if message metadata is added to the HTTP headers of // the delivered message. .setWriteMetadata(true) .build(); PushConfig pushConfig = PushConfig.newBuilder().setPushEndpoint(pushEndpoint).setNoWrapper(noWrapper).build(); // Create a push subscription with default acknowledgement deadline of 10 seconds. // Messages not successfully acknowledged within 10 seconds will get resent by the server. Subscription subscription = subscriptionAdminClient.createSubscription(subscriptionName, topicName, pushConfig, 10); System.out.println("Created push subscription: " + subscription.getName()); } } } ``` ### C++ Before trying this sample, follow the C++ setup instructions in [Quickstart: Using Client Libraries](https://docs.cloud.google.com/pubsub/docs/create-topic-client-libraries). For more information, see the [Pub/Sub C++ API reference documentation](https://googleapis.dev/cpp/google-cloud-pubsub/latest/). ```c++ namespace pubsub = ::google::cloud::pubsub; namespace pubsub_admin = ::google::cloud::pubsub_admin; [](pubsub_admin::SubscriptionAdminClient client, std::string const& project_id, std::string const& topic_id, std::string const& subscription_id, std::string const& endpoint) { google::pubsub::v1::Subscription request; request.set_name( pubsub::Subscription(project_id, subscription_id).FullName()); request.set_topic(pubsub::Topic(project_id, topic_id).FullName()); request.mutable_push_config()->set_push_endpoint(endpoint); request.mutable_push_config()->mutable_no_wrapper()->set_write_metadata( true); auto sub = client.CreateSubscription(request); if (sub.status().code() == google::cloud::StatusCode::kAlreadyExists) { std::cout << "The subscription already exists\n"; return; } if (!sub) throw std::move(sub).status(); std::cout << "The subscription was successfully created: " << sub->DebugString() << "\n"; } ``` ### Go The following sample uses the major version of the Go Pub/Sub client library (v2). If you are still using the v1 library, see [the migration guide to v2](https://github.com/googleapis/google-cloud-go/blob/main/pubsub/MIGRATING.md). To see a list of v1 code samples, see [the deprecated code samples](https://docs.cloud.google.com/pubsub/docs/samples.md.txt?language=golang&text=deprecated). Before trying this sample, follow the Go setup instructions in [Quickstart: Using Client Libraries](https://docs.cloud.google.com/pubsub/docs/create-topic-client-libraries). For more information, see the [Pub/Sub Go API reference documentation](https://pkg.go.dev/cloud.google.com/go/pubsub/v2). ```go import ( "context" "fmt" "io" "cloud.google.com/go/pubsub/v2" "cloud.google.com/go/pubsub/v2/apiv1/pubsubpb" ) // createPushNoWrapperSubscription creates a push subscription where messages are delivered in the HTTP body. func createPushNoWrapperSubscription(w io.Writer, projectID, topic, subscription, endpoint string) error { // projectID := "my-project-id" // topic := "projects/my-project-id/topics/my-topic" // subscription := "projects/my-project/subscriptions/my-sub" // endpoint := "https://my-test-project.appspot.com/push" ctx := context.Background() client, err := pubsub.NewClient(ctx, projectID) if err != nil { return fmt.Errorf("pubsub.NewClient: %w", err) } defer client.Close() pbSubscription := &pubsubpb.Subscription{ Name: subscription, Topic: topic, AckDeadlineSeconds: 10, PushConfig: &pubsubpb.PushConfig{ PushEndpoint: endpoint, Wrapper: &pubsubpb.PushConfig_NoWrapper_{ NoWrapper: &pubsubpb.PushConfig_NoWrapper{ // Determines if message metadata is added to the HTTP headers of // the delivered message. WriteMetadata: true, }, }, }, } sub, err := client.SubscriptionAdminClient.CreateSubscription(ctx, pbSubscription) if err != nil { return fmt.Errorf("CreateSubscription: %w", err) } fmt.Fprintf(w, "Created push no wrapper subscription: %v\n", sub) return nil } ``` ### Node.js Before trying this sample, follow the Node.js setup instructions in [Quickstart: Using Client Libraries](https://docs.cloud.google.com/pubsub/docs/create-topic-client-libraries). For more information, see the [Pub/Sub Node.js API reference documentation](https://googleapis.dev/nodejs/pubsub/latest). ```go /** * TODO(developer): Uncomment these variables before running the sample. */ // const pushEndpoint = 'YOUR_ENDPOINT_URL'; // const topicNameOrId = 'YOUR_TOPIC_NAME_OR_ID'; // const subscriptionNameOrId = 'YOUR_SUBSCRIPTION_NAME_OR_ID'; // Imports the Google Cloud client library const {PubSub} = require('@google-cloud/pubsub'); // Creates a client; cache this for further use const pubSubClient = new PubSub(); async function createPushSubscriptionNoWrapper( pushEndpoint, topicNameOrId, subscriptionNameOrId, ) { const options = { pushConfig: { // Set to an HTTPS endpoint of your choice. If necessary, register // (authorize) the domain on which the server is hosted. pushEndpoint, // When true, writes the Pub/Sub message metadata to // `x-goog-pubsub-:` headers of the HTTP request. Writes the // Pub/Sub message attributes to `:` headers of the HTTP request. noWrapper: { writeMetadata: true, }, }, }; await pubSubClient .topic(topicNameOrId) .createSubscription(subscriptionNameOrId, options); console.log(`Subscription ${subscriptionNameOrId} created.`); } ``` ### Node.js Before trying this sample, follow the Node.js setup instructions in [Quickstart: Using Client Libraries](https://docs.cloud.google.com/pubsub/docs/create-topic-client-libraries). For more information, see the [Pub/Sub Node.js API reference documentation](https://googleapis.dev/nodejs/pubsub/latest). ```go /** * TODO(developer): Uncomment these variables before running the sample. */ // const pushEndpoint = 'YOUR_ENDPOINT_URL'; // const topicNameOrId = 'YOUR_TOPIC_NAME_OR_ID'; // const subscriptionNameOrId = 'YOUR_SUBSCRIPTION_NAME_OR_ID'; // Imports the Google Cloud client library import {PubSub, CreateSubscriptionOptions} from '@google-cloud/pubsub'; // Creates a client; cache this for further use const pubSubClient = new PubSub(); async function createPushSubscriptionNoWrapper( pushEndpoint: string, topicNameOrId: string, subscriptionNameOrId: string, ) { const options: CreateSubscriptionOptions = { pushConfig: { // Set to an HTTPS endpoint of your choice. If necessary, register // (authorize) the domain on which the server is hosted. pushEndpoint, // When true, writes the Pub/Sub message metadata to // `x-goog-pubsub-:` headers of the HTTP request. Writes the // Pub/Sub message attributes to `:` headers of the HTTP request. noWrapper: { writeMetadata: true, }, }, }; await pubSubClient .topic(topicNameOrId) .createSubscription(subscriptionNameOrId, options); console.log(`Subscription ${subscriptionNameOrId} created.`); } ``` ## Set a content-type header in your message After enabling payload unwrapping, Pub/Sub doesn't automatically set a media type header field in your request. If you don't explicitly set a `Content-Type` header field, the web server processing your request might set a [default value of `application/octet-stream`](https://www.rfc-editor.org/rfc/rfc7231#section-3.1.1.5) or interpret the request in an unexpected manner. If you require a `Content-Type` header, make sure that you explicitly declare it at publish time for every individual published message. To do this, you must first enable **Write metadata** . The result of enabling **Write metadata** is shown in the [provided examples](https://docs.cloud.google.com/pubsub/docs/payload-unwrapping.md.txt#message-examples). ## What's next - If you're having issues with payload unwrapping, see [Troubleshoot payload unwrapping](https://docs.cloud.google.com/pubsub/docs/payload-unwrapping-troubleshooting.md.txt).