# Managed Service for Apache Spark documentation [Read product documentation](https://docs.cloud.google.com/managed-spark/docs/overview/key-concepts.md.txt) Managed Service for Apache Spark on clusters lets you take advantage of open source data tools for batch processing, querying, streaming, and machine learning. Managed Service for Apache Spark automation helps you create clusters quickly, manage them easily, and save money by turning clusters off when you don't need them. With less time and money spent on administration, you can focus on your jobs and your data. Use Managed Service for Apache Spark serverless to run Spark batch workloads without provisioning and managing your own cluster. Specify workload parameters, and then submit the workload to the Managed Service for Apache Spark service. The service will run the workload on a managed compute infrastructure, autoscaling resources as needed. Managed Service for Apache Spark charges apply only to the time when the workload is executing. [Go to the Managed Service for Apache Spark product page for more.](https://cloud.google.com/products/managed-service-for-apache-spark) ## Documentation resources Find quickstarts and guides, review key references, and get help with common issues. ### Guides - Quickstarts: [Create a cluster](https://docs.cloud.google.com/managed-spark/docs/quickstarts/update-cluster-console), [Submit an Apache Spark batch workload](https://docs.cloud.google.com/managed-spark/docs/quickstarts/spark-batch.md.txt), or [Run a Spark job on Google Kubernetes Engine](https://docs.cloud.google.com/managed-spark/docs/guides/dpgke/quickstarts/gke-quickstart-create-cluster.md.txt) - [Overview of Managed Service for Apache Spark Workflow Templates](https://docs.cloud.google.com/managed-spark/docs/concepts/workflows/overview.md.txt) - [Create a Managed Service for Apache Spark Custom Image](https://docs.cloud.google.com/managed-spark/docs/guides/images.md.txt) - [Write a MapReduce job with the BigQuery connector](https://docs.cloud.google.com/managed-spark/docs/tutorials/bigquery-connector-mapreduce-example.md.txt) - [Use the Cloud Storage connector with Apache Spark](https://docs.cloud.google.com/managed-spark/docs/tutorials/gcs-connector-spark-tutorial.md.txt) ### Reference - [REST API](https://docs.cloud.google.com/managed-spark/docs/reference/rest.md.txt) - [RPC API](https://docs.cloud.google.com/managed-spark/docs/reference/rpc.md.txt) - [Dataproc Client Libraries](https://docs.cloud.google.com/managed-spark/docs/reference/libraries.md.txt) - [Managed Service for Apache Spark \& Cloud SDK](https://docs.cloud.google.com/managed-spark/docs/gcloud-installation.md.txt) ### Resources - [Best practices](https://cloud.google.com/blog/topics/developers-practitioners/best-practices-guide) - [Pricing](https://docs.cloud.google.com/dataproc/pricing) - [Release notes](https://docs.cloud.google.com/managed-spark/docs/release-notes.md.txt) - [Diagnose Managed Service for Apache Spark clusters](https://docs.cloud.google.com/managed-spark/docs/support/diagnose-command) - [Managed Service for Apache Spark Quotas](https://docs.cloud.google.com/managed-spark/quotas.md.txt) - [Get support](https://docs.cloud.google.com/managed-spark/docs/support/getting-support.md.txt) - ## Related resources Training and tutorials Use cases Code samples Explore self-paced training, use cases, reference architectures, and code samples with examples of how to use and connect Google Cloud services. Tutorial Training and tutorials ### [Run a Spark job on Google Kubernetes Engine](https://docs.cloud.google.com/managed-spark/docs/guides/dpgke/quickstarts/gke-quickstart-create-cluster.md.txt) Submit Spark jobs to a running Google Kubernetes Engine cluster from the Dataproc Jobs API. Training Training and tutorials ### [Introduction to Cloud Dataproc: Hadoop and Spark on Google Cloud](https://www.cloudskillsboost.google/focuses/672?parent=catalog) This course features a combination of lectures, demos, and hands-on labs to create a Dataproc cluster, submit a Spark job, and then shut down the cluster. Training Training and tutorials ### [Machine Learning with Spark on Dataproc](https://www.cloudskillsboost.google/focuses/3390?parent=catalog) This course features a combination of lectures, demos, and hands-on labs to implement logistic regression using a machine learning library for Apache Spark running on a Dataproc cluster to develop a model for data from a multivariable dataset. Solutions Use cases ### [Workflow scheduling solutions](https://docs.cloud.google.com/managed-spark/docs/concepts/workflows/workflow-schedule-solutions.md.txt) Schedule workflows on Google Cloud. Use case Use cases ### [Migrate HDFS Data from On-Premises to Google Cloud](https://docs.cloud.google.com/solutions/migration/hadoop/hadoop-gcp-migration-data) How to move data from on-premises Hadoop Distributed File System (HDFS) to Google Cloud. Best practice Use cases ### [Manage Java and Scala dependencies for Apache Spark](https://docs.cloud.google.com/managed-spark/docs/guides/manage-spark-dependencies.md.txt) Recommended approaches to including dependencies when you submit a Spark job to a Managed Service for Apache Spark cluster. Code sample Code Samples ### [Python API samples](https://github.com/googleapis/python-managed-spark/tree/master/samples) Call Dataproc APIs from Python. Code sample Code Samples ### [Java API samples](https://github.com/GoogleCloudPlatform/java-docs-samples/tree/main/dataproc) Call Dataproc APIs from Java. Code sample Code Samples ### [Node.js API samples](https://github.com/GoogleCloudPlatform/nodejs-docs-samples/tree/main/dataproc) Call Dataproc APIs from Node.js. Code sample Code Samples ### [Go API samples](https://github.com/GoogleCloudPlatform/golang-samples/tree/master/dataproc) Call Dataproc APIs from Go. ## Related videos