# Google Cloud Dataproc v1 API - Class PySparkJob (5.27.0)

5.27.0 (latest) 5.26.0 5.25.0 5.24.0 5.23.0 5.22.0 5.21.0 5.20.0 5.19.0 5.18.0 5.17.0 5.16.0 5.15.0 5.14.0 5.13.0 5.12.0 5.11.0 5.10.0 5.9.0 5.8.0 5.7.0 5.6.0 5.5.0 5.4.0 5.3.0 5.2.0 5.1.0 5.0.0 4.0.0 3.4.0 3.3.0 3.2.0 3.1.0

    public sealed class PySparkJob : IMessage<PySparkJob>, IEquatable<PySparkJob>, IDeepCloneable<PySparkJob>, IBufferMessage, IMessage

Reference documentation and code samples for the Google Cloud Dataproc v1 API class PySparkJob.

A Dataproc job for running
[Apache
PySpark](https://spark.apache.org/docs/0.9.0/python-programming-guide.html)
applications on YARN.

## Inheritance

[object](https://learn.microsoft.com/dotnet/api/system.object) \> PySparkJob

## Implements

[IMessage](https://cloud.google.com/dotnet/docs/reference/Google.Protobuf/latest/Google.Protobuf.IMessage-1.html)[PySparkJob](https://docs.cloud.google.com/dotnet/docs/reference/Google.Cloud.Dataproc.V1/latest/Google.Cloud.Dataproc.V1.PySparkJob), [IEquatable](https://learn.microsoft.com/dotnet/api/system.iequatable-1)[PySparkJob](https://docs.cloud.google.com/dotnet/docs/reference/Google.Cloud.Dataproc.V1/latest/Google.Cloud.Dataproc.V1.PySparkJob), [IDeepCloneable](https://cloud.google.com/dotnet/docs/reference/Google.Protobuf/latest/Google.Protobuf.IDeepCloneable-1.html)[PySparkJob](https://docs.cloud.google.com/dotnet/docs/reference/Google.Cloud.Dataproc.V1/latest/Google.Cloud.Dataproc.V1.PySparkJob), [IBufferMessage](https://cloud.google.com/dotnet/docs/reference/Google.Protobuf/latest/Google.Protobuf.IBufferMessage.html), [IMessage](https://cloud.google.com/dotnet/docs/reference/Google.Protobuf/latest/Google.Protobuf.IMessage.html)

## Inherited Members

[object.GetHashCode()](https://learn.microsoft.com/dotnet/api/system.object.gethashcode) [object.GetType()](https://learn.microsoft.com/dotnet/api/system.object.gettype) [object.ToString()](https://learn.microsoft.com/dotnet/api/system.object.tostring)

## Namespace

[Google.Cloud.Dataproc.V1](https://docs.cloud.google.com/dotnet/docs/reference/Google.Cloud.Dataproc.V1/latest/Google.Cloud.Dataproc.V1)

## Assembly

Google.Cloud.Dataproc.V1.dll

## Constructors

### PySparkJob()

    public PySparkJob()

### PySparkJob(PySparkJob)

    public PySparkJob(PySparkJob other)

| **Parameter** ||
|---|---|
| **Name** | **Description** |
| `other` | `https://docs.cloud.google.com/dotnet/docs/reference/Google.Cloud.Dataproc.V1/latest/Google.Cloud.Dataproc.V1.PySparkJob` |

## Properties

### ArchiveUris

    public RepeatedField<string> ArchiveUris { get; }

Optional. HCFS URIs of archives to be extracted into the working directory
of each executor. Supported file types:
.jar, .tar, .tar.gz, .tgz, and .zip.

| **Property Value** ||
|---|---|
| **Type** | **Description** |
| `https://cloud.google.com/dotnet/docs/reference/Google.Protobuf/latest/Google.Protobuf.Collections.RepeatedField-1.htmlhttps://learn.microsoft.com/dotnet/api/system.string` |   |

### Args

    public RepeatedField<string> Args { get; }

Optional. The arguments to pass to the driver. Do not include arguments,
such as `--conf`, that can be set as job properties, since a collision may
occur that causes an incorrect job submission.

| **Property Value** ||
|---|---|
| **Type** | **Description** |
| `https://cloud.google.com/dotnet/docs/reference/Google.Protobuf/latest/Google.Protobuf.Collections.RepeatedField-1.htmlhttps://learn.microsoft.com/dotnet/api/system.string` |   |

### FileUris

    public RepeatedField<string> FileUris { get; }

Optional. HCFS URIs of files to be placed in the working directory of
each executor. Useful for naively parallel tasks.

| **Property Value** ||
|---|---|
| **Type** | **Description** |
| `https://cloud.google.com/dotnet/docs/reference/Google.Protobuf/latest/Google.Protobuf.Collections.RepeatedField-1.htmlhttps://learn.microsoft.com/dotnet/api/system.string` |   |

### JarFileUris

    public RepeatedField<string> JarFileUris { get; }

Optional. HCFS URIs of jar files to add to the CLASSPATHs of the
Python driver and tasks.

| **Property Value** ||
|---|---|
| **Type** | **Description** |
| `https://cloud.google.com/dotnet/docs/reference/Google.Protobuf/latest/Google.Protobuf.Collections.RepeatedField-1.htmlhttps://learn.microsoft.com/dotnet/api/system.string` |   |

### LoggingConfig

    public LoggingConfig LoggingConfig { get; set; }

Optional. The runtime log config for job execution.

| **Property Value** ||
|---|---|
| **Type** | **Description** |
| `https://docs.cloud.google.com/dotnet/docs/reference/Google.Cloud.Dataproc.V1/latest/Google.Cloud.Dataproc.V1.LoggingConfig` |   |

### MainPythonFileUri

    public string MainPythonFileUri { get; set; }

Required. The HCFS URI of the main Python file to use as the driver. Must
be a .py file.

| **Property Value** ||
|---|---|
| **Type** | **Description** |
| `https://learn.microsoft.com/dotnet/api/system.string` |   |

### Properties

    public MapField<string, string> Properties { get; }

Optional. A mapping of property names to values, used to configure PySpark.
Properties that conflict with values set by the Dataproc API might be
overwritten. Can include properties set in
/etc/spark/conf/spark-defaults.conf and classes in user code.

| **Property Value** ||
|---|---|
| **Type** | **Description** |
| `https://cloud.google.com/dotnet/docs/reference/Google.Protobuf/latest/Google.Protobuf.Collections.MapField-2.htmlhttps://learn.microsoft.com/dotnet/api/system.stringhttps://learn.microsoft.com/dotnet/api/system.string` |   |

### PythonFileUris

    public RepeatedField<string> PythonFileUris { get; }

Optional. HCFS file URIs of Python files to pass to the PySpark
framework. Supported file types: .py, .egg, and .zip.

| **Property Value** ||
|---|---|
| **Type** | **Description** |
| `https://cloud.google.com/dotnet/docs/reference/Google.Protobuf/latest/Google.Protobuf.Collections.RepeatedField-1.htmlhttps://learn.microsoft.com/dotnet/api/system.string` |   |