This page shows the required configuration for exposing an App Engine flexible environment service only on its internal IP address. By default, flexible environment services receive both an internal IP address and an ephemeral external IP address when they are first deployed. The ephemeral external IP address allows your service to send requests to App Engine services with custom domains, and resources on the internet. Ephemeral external IP addresses incur [costs](https://docs.cloud.google.com/vpc/network-pricing#ipaddress). If your service doesn't require an external IP address, you can prevent your service from sending requests to resources on the internet and reduce costs by limiting your service to using only its internal IP address. This doesn't prevent your service from receiving requests from the internet or App Engine services with custom domains, because the fully qualified domain name of your service is still externally visible. ## Limitations Disabling ephemeral external IP addresses has the following limitations: - **Services with custom domains:** To send requests to App Engine services with custom domains you must [configure Cloud NAT](https://docs.cloud.google.com/nat/docs/set-up-manage-network-address-translation.md.txt). - **External resources:** To send requests to external resources you must [configure Cloud NAT](https://docs.cloud.google.com/nat/docs/set-up-manage-network-address-translation.md.txt). - **Private Google Access dependency:** Instances with IP mode set to `internal` require [Private Google Access](https://docs.cloud.google.com/vpc/docs/private-google-access.md.txt#pga-supported) on the target subnetwork. - **Legacy networks:** [Legacy networks](https://docs.cloud.google.com/vpc/docs/legacy.md.txt) can't use Private Google Access, and thus can't disable ephemeral external IP addresses. ## Before you begin To deploy flexible environment apps without external IP addresses, you must enable Private Google Access on the target subnetwork. - To learn about Private Google Access, see the [Overview](https://docs.cloud.google.com/vpc/docs/private-google-access.md.txt#pga-supported). - For step-by-step configuration instructions, see [Enabling Private Google Access](https://docs.cloud.google.com/vpc/docs/configure-private-google-access.md.txt#enabling-pga). ### Prepare your Shared VPC network If you use Shared VPC, follow these steps to prepare your Shared VPC network for further configuration. 1. Verify that you have a route that is compatible with Private Google Access. Typically the [default route](https://docs.cloud.google.com/vpc/docs/routes.md.txt#routingpacketsinternet) of a network is compatible with Private Google Access. For other routes, confirm that the route is configured as follows. ```sh Network: SHARED_VPC_NETWORK_NAME Destination IP address range: 0.0.0.0/0 Instance tags: INSTANCE_TAGS Next hop: DEFAULT_INTERNET_GATEWAY ``` Replace the following: - `SHARED_VPC_NETWORK_NAME`: The name of your Shared VPC network. - `INSTANCE_TAGS`: If you don't use any instance tags, don't specify anything in this field. If you do use instance tags, include `aef-instances` in your list of instance tags. - `DEFAULT_INTERNET_GATEWAY`: The default internet gateway. To learn more about compatible routes for Private Google Access, see the Private Google Access documentation on [routing options](https://docs.cloud.google.com/vpc/docs/configure-private-google-access.md.txt#config-routing). 2. Verify that you have a firewall rule that is compatible with Private Google Access. The firewall rule must be configured as follows. ```sh Network: SHARED_VPC_NETWORK_NAME Destination IP address range: 0.0.0.0/0 Destination filter: IP ranges Direction of traffic: Egress Attach on match: Allow Instance tags: INSTANCE_TAGS ``` Replace the following: - `SHARED_VPC_NETWORK_NAME`: The name of your Shared VPC network. - `INSTANCE_TAGS`: If you don't use any instance tags, don't specify anything in this field. If you do use instance tags, include `aef-instances` in your list of instance tags. To learn more about compatible firewall rules for Private Google Access, see the Private Google Access documentation on [firewall configuration](https://docs.cloud.google.com/vpc/docs/configure-private-google-access.md.txt#config-firewall). ## Configure your service to use only its internal IP address 1. Update the Google Cloud CLI. This ensures that you're using a version of the gcloud CLI that supports private IP addresses for flexible environment apps. ```bash gcloud components update ``` 2. In your `app.yaml` file, add the `instance_ip_mode` field to the [`network` section](https://docs.cloud.google.com/appengine/docs/flexible/reference/app-yaml.md.txt#network_settings) and set it to `internal`. - If your `app.yaml` file already has a `network` section, add the following line inside the `network` section: ```bash instance_ip_mode: internal ```
- If your `app.yaml` file doesn't have a `network` section, create the section and specify the instance IP mode by adding the following lines: ```bash network: instance_ip_mode: internal ```
Save these changes. 3. Deploy the service. ```bash gcloud beta app deploy ``` 4. Verify configuration by checking the Instances page of the Google Cloud console. [Go to Instances](https://console.cloud.google.com/appengine/instances) Go to the **Instances** table after the **Summary** chart. In the **External IP** column, confirm that there is no IP address listed. The absence of an IP address in this column means that your instance has no external IP address. Even though this field is empty, your instance still has an internal IP address. ## Send external requests without an external IP address If your service sends requests to the internet but you want to limit it to using only its internal IP address, you can use [Cloud NAT](https://docs.cloud.google.com/nat/docs/overview.md.txt) to create a gateway. Your service can send outbound requests through the Cloud NAT gateway using reserved static external IP addresses instead of default ephemeral external IP addresses. Configure your service to use `instance_ip_mode: internal` with Cloud NAT and dedicated static external IP addresses in the following situations: - You connect to an external database or API that requires an IP address allowlist or firewall rule. - You connect to reputation-sensitive external endpoints, such as payment gateways, hosted databases, or partner APIs protected by third-party web application firewalls (WAFs). Prior or noisy traffic on shared ephemeral IP address pools can degrade IP reputation and cause external WAFs to block or throttle your requests with `HTTP 403` errors or connection timeouts. - You run authorized network scanning, port enumeration, host fingerprinting, vulnerability probing, web scraping, or load testing workloads. > [!WARNING] > **Warning:** Don't run network scanning, port enumeration, host fingerprinting, vulnerability probing, web scraping, or load testing over default ephemeral external IP addresses. Unisolated scanning traffic degrades IP reputation, can disrupt service for other customers in violation of the [Google Cloud Acceptable Use Policy](https://cloud.google.com/terms/aup), and might cause Google to throttle or suspend your workload. To route outbound internet requests through Cloud NAT: 1. Follow the steps in the section [Configure your service to use only its internal IP address](https://docs.cloud.google.com/appengine/docs/flexible/disable-external-ip.md.txt#limit-to-internal). 2. Follow the steps to [Configure Cloud NAT](https://docs.cloud.google.com/nat/docs/set-up-manage-network-address-translation.md.txt) and assign reserved static external IP addresses to the gateway. If your service scales to many instances or opens many concurrent connections, such as during authorized network scanning, assign multiple static IP addresses to Cloud NAT. You can also increase the minimum ports per instance or enable dynamic port allocation to prevent port exhaustion. Learn more about how Cloud NAT gateways use [IP addresses and ports](https://docs.cloud.google.com/nat/docs/ports-and-addresses.md.txt). To learn more about this approach, see the Cloud Architecture Center documentation on [Deploying Cloud NAT for fetching](https://docs.cloud.google.com/architecture/building-internet-connectivity-for-private-vms#deploying_cloud_nat_for_fetching).