Managed Service for Apache Spark provides a [gcloud CLI dataproc clusters diagnose](https://docs.cloud.google.com/sdk/gcloud/reference/dataproc/clusters/diagnose)
command to help you troubleshoot Managed Service for Apache Spark on GKE cluster and job issues. This
command gathers and archives cluster-related configuration files, logs, and outputs
into an archive file. and then uploads the archive to the Cloud Storage
staging bucket you specified when you
[created your Managed Service for Apache Spark on GKE cluster](https://docs.cloud.google.com/managed-spark/docs/guides/dpgke/quickstarts/dataproc-gke-quickstart-create-cluster#create_a_on_cluster).

## Diagnose archive file

The following tables list metrics and other information included in the
`dataproc clusters diagnose` command archive file.

### System information

| Item | Archive location |
|---|---|
| **GKE node metrics where virtual Managed Service for Apache Spark on GKE pods run:** |
| - CPU usage - Memory usage | `/system/NODE_NAME.json` |
| **Network metrics and file system status of running pods:** |
| - CPU usage - Memory usage - Network status - Filesystem status | `/system/POD_NAME.json` |

### Configuration information

| Item | Archive location |
|---|---|
| Cluster configmap | `/conf/configmap` |
| Kubernetes deployment | `/conf/deployment` |
| [Role Based Access Control (RBAC)](https://docs.cloud.google.com/kubernetes-engine/docs/how-to/google-groups-rbac.md.txt) | - `/conf/role` - `/conf/rolebind` - `/conf/serviceaccount` |

### Logs

| Item | Archive location |
|---|---|
| Agent log | `/logs/agent.log` |
| Spark engine log | `/logs/sparkengine.log` |
| Spark driver running and completed job logs over the last 24 hours | `/logs/DRIVER_ID` |

## Job and pod information

| Item | Archive location |
|---|---|
| JobAttempt object | `/jobattempts` |
| Kubernetes Pod object | `/pods` |

## What's next

- See [Diagnose Managed Service for Apache Spark clusters](https://docs.cloud.google.com/managed-spark/docs/support/diagnose-command#sharing_the_data_gathered_by_diagnose).