Troubleshooting Looker (Google Cloud core) networking issues

OIDC authentication, timeout errors, Private Service Connect, and private services access.

If your Looker (Google Cloud core) instance is failing to connect to external services, identity providers, or database backends, use this guide to diagnose and resolve common networking issues. You can troubleshoot OpenID Connect (OIDC) authentication failures, 504 and HttpClient:ConnectTimeoutError timeouts, and route propagation errors for Private Service Connect (PSC) and private services access.

OpenID Connect authentication failure with controlled egress

When you set up OpenID Connect authentication on a Looker (Google Cloud core) instance that's configured with Private Service Connect and controlled egress enabled, authentication might fail.

Symptoms

If authentication has failed, you might experience either of the following symptoms:

  • The Looker /openidconnect endpoint hangs for approximately 60 seconds in the browser before displaying an error or timing out with a 504 Upstream Request Timeout or 504 Gateway Timeout error.

  • Your identity provider displays an error message indicating that the redirect URI doesn't match, such as:

    AADSTS50011: The redirect URI specified in the request does not match the
redirect URIs configured for the application

    This error message is misleading because the redirect URI that you configured in your identity provider might be correct. The actual failure occurs when Looker (Google Cloud core) cannot contact the token endpoint of the identity provider during the backend exchange.

Cause

The automated provisioning workflow for controlled egress occasionally fails to create the necessary Secure Web Proxy network route (looker-swp-route) in the Looker (Google Cloud core) tenant project.

When this route is missing, backend requests from Looker (Google Cloud core) to the identity provider's token endpoint time out because the instance cannot route outbound traffic to the public internet through the Secure Web Proxy.

Resolution

To resolve this issue, follow these steps to reset the controlled egress configuration to force the system to recreate the missing network route:

  1. In the Google Cloud console, go to the Looker page:

    Go to Looker

  2. Click the name of the instance that you want to update.

  3. Click Edit.

  4. Expand the Connections section and go to the Configure outbound connections section.

  5. Record all existing fully qualified domain names (FQDNs) listed in the Global FQDN section (such as your identity provider's endpoints).

  6. Delete all FQDNs that are listed in the Global FQDN section.

  7. Clear the Enable Looker to connect to external services checkbox to disable controlled egress.

  8. Click Save.

  9. Wait for the instance update to finish and for the configuration changes to clear.

  10. Click Edit again.

  11. Expand the Connections section and select the Enable Looker to connect to external services checkbox.

  12. In the Global FQDN section, re-add the required FQDNs for your identity provider and any other external services.

  13. Click Save.

Resetting controlled egress triggers the automated provisioning process to recreate the missing looker-swp-route network route. After the route is created, outbound backend requests to the identity provider can be completed.

504 Upstream Request Timeout error

When authenticating with an external service or identity provider on a Looker (Google Cloud core) instance with controlled egress enabled, the request might fail with a timeout error.

Symptoms

You receive a 504 Upstream Request Timeout error in your browser after returning from an identity provider's login page or when connecting to an external service.

Cause

A 504 Upstream Request Timeout error indicates that the Looker (Google Cloud core) instance cannot reach the external domain through controlled egress. Unlike missing route errors, a 504 error typically means that the network path exists, but a required domain is missing from the Global FQDN allowlist and is being blocked, or the request is timing out at the gateway.

Resolution

To resolve this issue, follow these steps:

  1. Verify required FQDNs: In the Google Cloud console, verify that all domains that are required by your external service or identity provider are added to the Global FQDN list. OIDC authentication flows often require multiple endpoints (such as authorization, token, and user information endpoints) that might use different domains.
  2. Capture a network trace: If the error persists, capture a browser network trace (HAR file) while reproducing the issue. Search the trace for 504 status codes or failed HTTP requests to identify any domains that were contacted during the flow but aren't included in your Global FQDN list.
  3. Reset configuration: If the required FQDNs are present but requests continue to time out, follow the steps in the Resolution for OpenID Connect authentication failures to reset the controlled egress configuration.

HttpClient:ConnectTimeoutError execution expired

When testing an OpenID Connect connection or other external network connections in the Looker Admin panel, the connection test might fail with a timeout error.

Symptoms

When you run a connection test in the Looker Admin panel (such as on the OpenID Connect Authentication page), Looker returns the following error:

HttpClient:ConnectTimeoutError execution expired

Cause

This error indicates that the Looker (Google Cloud core) instance lacks the necessary network path that it needs to communicate with the external service or identity provider. This issue typically occurs when either of the following conditions is met:

  • Missing route propagation: Cloud Router is using default settings and isn't propagating VPC network routes (such as custom advertised routes) to the Looker (Google Cloud core) instance.
  • Missing internet NEG: An internet network endpoint group (NEG) isn't configured in your VPC network to route traffic to the external service.

Resolution

To resolve this issue, follow these steps:

  1. Configure Cloud Router route propagation: Verify that Cloud Router is configured to propagate the necessary VPC network routes to your Looker (Google Cloud core) instance by setting up custom advertised routes.
  2. Configure an internet NEG: Set up an internet NEG in your VPC network to allow the Looker (Google Cloud core) instance to route egress traffic to the external identity provider or service.
  3. Verify private services access: For instances that use private services access, ensure that your VPC network is properly peered with Google services. For more information, see Access external services using private services access.

Southbound Private Service Connect connection issues

For Looker (Google Cloud core) instances that use Private Service Connect for outbound connections to database backends or published services, you might encounter connection errors even when the endpoint status is Accepted.

Common issues include the following:

  • Hostname resolution errors: If Looker (Google Cloud core) displays an Unknown host error when testing a connection, verify that the configured hostname matches the DNS record in your private network and that the backend load balancer is healthy.
  • Connection timeouts: If connections time out, check your VPC firewall rules to verify that ingress traffic is allowed from the Private Service Connect NAT subnet to your load balancer's backends.
  • Pending or Closed status: If the service attachment status is Pending or Closed, check that the consumer project is in the service attachment's allowlist and that mandatory TCP health checks are configured.

For detailed diagnostic workflows and a decision tree, see Troubleshooting southbound Private Service Connect connections.

Private services access route and connectivity issues

For Looker (Google Cloud core) instances that use private services access, outbound communication to external services requires proper route propagation and network infrastructure:

  • Route propagation: Verify that Cloud Router is configured to propagate VPC network routes to the Looker (Google Cloud core) instance, such as custom advertised routes.
  • Internet network endpoint groups: If your instance needs to reach external endpoints over private services access, ensure that an internet network endpoint group (NEG) is configured in your VPC network.

If these routes or endpoint groups are missing, connection tests to external endpoints fail with an HttpClient:ConnectTimeoutError execution expired error.

For more information, see Access external services using private services access.

Diagnostic tools and logs

Use the following tools and logs to help diagnose networking issues:

  • Cloud Logging: View instance-level logs in Cloud Logging to identify connection failures and system events. For more information, see View instance logs.
  • Looker System Activity: If you can sign in as an administrator, review the User Activity dashboard in System Activity. This dashboard displays recent login failures, the authentication method that was used, and error messages. For more information, see User Activity dashboard.
  • VPC Flow Logs: Check VPC Flow Logs to determine whether network traffic from Looker (Google Cloud core) reaches your destination subnet or is blocked by firewall rules.
  • Cloud Customer Care: If you completed the troubleshooting steps and the issue persists, contact Cloud Customer Care for assistance.