Class PySparkNotebookBatch (5.28.0)

5.28.0 (latest) 5.27.0 5.26.0 5.25.0 5.24.0 5.23.0 5.22.0 5.21.0 5.20.0 5.18.1 5.17.1 5.16.0 5.15.1 5.14.0 5.13.0 5.12.0 5.10.2 5.9.3 5.8.0 5.7.0 5.6.0 5.5.1 5.4.3 5.3.0 5.2.0 5.1.0 5.0.3 4.0.3 3.3.2 3.2.0 3.1.1 3.0.0 2.6.2 2.5.0 2.4.0 2.3.1 2.2.0 2.0.2 1.1.3 1.0.1 0.8.2 0.7.0 0.6.1 0.5.0

PySparkNotebookBatch(mapping=None, *, ignore_unknown_fields=False, **kwargs)

A configuration for running a PySpark Notebook batch workload.

Attributes
Name	Description
notebook_file_uri	str Required. The HCFS URI of the notebook file to execute.
params	MutableMapping[str, str] Optional. The parameters to pass to the notebook.
python_file_uris	MutableSequence[str] Optional. HCFS URIs of Python files to pass to the PySpark framework.
jar_file_uris	MutableSequence[str] Optional. HCFS URIs of jar files to be added to the Spark CLASSPATH.
file_uris	MutableSequence[str] Optional. HCFS URIs of files to be placed in the working directory of each executor
archive_uris	MutableSequence[str] Optional. HCFS URIs of archives to be extracted into the working directory of each executor. Supported file types: .jar, .tar, .tar.gz, .tgz, and .zip.

Classes

ParamsEntry

ParamsEntry(mapping=None, *, ignore_unknown_fields=False, **kwargs)

The abstract base class for a message.

Parameters
Name	Description
kwargs	dict Keys and values corresponding to the fields of the message.
mapping	Union[dict, .Message] A dictionary or message to be used to determine the values for this message.
ignore_unknown_fields	Optional(bool) If True, do not raise errors for unknown fields. Only applied if mapping is a mapping type or there are keyword parameters.