Reference documentation and code samples for the Google Cloud Dataplex V1 Client class DataQualitySpec.
DataQualityScan related setting.
Generated from protobuf message google.cloud.dataplex.v1.DataQualitySpec
Namespace
Google \ Cloud \ Dataplex \ V1Methods
__construct
Constructor.
| Parameters | |
|---|---|
| Name | Description |
data |
array
Optional. Data for populating the Message object. |
↳ rules |
array<DataQualityRule>
Required. The list of rules to evaluate against a data source. At least one rule is required. |
↳ sampling_percent |
float
Optional. The percentage of the records to be selected from the dataset for DataScan. * * Value can range between 0.0 and 100.0 with up to 3 significant decimal digits. * * Sampling is not applied if |
↳ row_filter |
string
Optional. A filter applied to all rows in a single DataScan job. The filter needs to be a valid SQL expression for a WHERE clause in GoogleSQL syntax. Example: col1 >= 0 AND col2 < 10 |
↳ post_scan_actions |
DataQualitySpec\PostScanActions
Optional. Actions to take upon job completion. |
↳ catalog_publishing_enabled |
bool
Optional. If set, the latest DataScan job result will be published as Dataplex Universal Catalog metadata. |
↳ enable_catalog_based_rules |
bool
Optional. If enabled, the data scan will retrieve rules defined in the dataplex-types.global.data-rules aspect on all paths of the catalog entry corresponding to the BigQuery table resource and all attached glossary terms. The path that data-rules aspect is attached on the table entry defines the column that the rule will be evaluated against. For glossary terms, the path that the terms are attached on the table entry defines the column that the rule will be evaluated against. At the start of scan execution, the rules reflect the latest state retrieved from the catalog entry and any updates on the rules thereafter are ignored for that execution. The updates will be reflected from the next execution. Rules defined in the datascan must be empty if this field is enabled. |
↳ filter |
string
Optional. Filter for selectively running a subset of rules. You can filter the request by the name or attribute key-value pairs defined on the rule. If not specified, all rules are run. The filter is applicable to both, the rules retrieved from catalog and explicitly defined rules in the scan. Please see filter syntax for more details. |
getRules
Required. The list of rules to evaluate against a data source. At least one rule is required.
| Returns | |
|---|---|
| Type | Description |
Google\Protobuf\RepeatedField<DataQualityRule> |
|
setRules
Required. The list of rules to evaluate against a data source. At least one rule is required.
| Parameter | |
|---|---|
| Name | Description |
var |
array<DataQualityRule>
|
| Returns | |
|---|---|
| Type | Description |
$this |
|
getSamplingPercent
Optional. The percentage of the records to be selected from the dataset for DataScan.
- Value can range between 0.0 and 100.0 with up to 3 significant decimal digits.
- Sampling is not applied if
sampling_percentis not specified, 0 or 100.
| Returns | |
|---|---|
| Type | Description |
float |
|
setSamplingPercent
Optional. The percentage of the records to be selected from the dataset for DataScan.
- Value can range between 0.0 and 100.0 with up to 3 significant decimal digits.
- Sampling is not applied if
sampling_percentis not specified, 0 or 100.
| Parameter | |
|---|---|
| Name | Description |
var |
float
|
| Returns | |
|---|---|
| Type | Description |
$this |
|
getRowFilter
Optional. A filter applied to all rows in a single DataScan job.
The filter needs to be a valid SQL expression for a WHERE clause in GoogleSQL syntax. Example: col1 >= 0 AND col2 < 10
| Returns | |
|---|---|
| Type | Description |
string |
|
setRowFilter
Optional. A filter applied to all rows in a single DataScan job.
The filter needs to be a valid SQL expression for a WHERE clause in GoogleSQL syntax. Example: col1 >= 0 AND col2 < 10
| Parameter | |
|---|---|
| Name | Description |
var |
string
|
| Returns | |
|---|---|
| Type | Description |
$this |
|
getPostScanActions
Optional. Actions to take upon job completion.
| Returns | |
|---|---|
| Type | Description |
DataQualitySpec\PostScanActions|null |
|
hasPostScanActions
clearPostScanActions
setPostScanActions
Optional. Actions to take upon job completion.
| Parameter | |
|---|---|
| Name | Description |
var |
DataQualitySpec\PostScanActions
|
| Returns | |
|---|---|
| Type | Description |
$this |
|
getCatalogPublishingEnabled
Optional. If set, the latest DataScan job result will be published as Dataplex Universal Catalog metadata.
| Returns | |
|---|---|
| Type | Description |
bool |
|
setCatalogPublishingEnabled
Optional. If set, the latest DataScan job result will be published as Dataplex Universal Catalog metadata.
| Parameter | |
|---|---|
| Name | Description |
var |
bool
|
| Returns | |
|---|---|
| Type | Description |
$this |
|
getEnableCatalogBasedRules
Optional. If enabled, the data scan will retrieve rules defined in the dataplex-types.global.data-rules aspect on all paths of the catalog entry corresponding to the BigQuery table resource and all attached glossary terms. The path that data-rules aspect is attached on the table entry defines the column that the rule will be evaluated against. For glossary terms, the path that the terms are attached on the table entry defines the column that the rule will be evaluated against. At the start of scan execution, the rules reflect the latest state retrieved from the catalog entry and any updates on the rules thereafter are ignored for that execution. The updates will be reflected from the next execution. Rules defined in the datascan must be empty if this field is enabled.
| Returns | |
|---|---|
| Type | Description |
bool |
|
setEnableCatalogBasedRules
Optional. If enabled, the data scan will retrieve rules defined in the dataplex-types.global.data-rules aspect on all paths of the catalog entry corresponding to the BigQuery table resource and all attached glossary terms. The path that data-rules aspect is attached on the table entry defines the column that the rule will be evaluated against. For glossary terms, the path that the terms are attached on the table entry defines the column that the rule will be evaluated against. At the start of scan execution, the rules reflect the latest state retrieved from the catalog entry and any updates on the rules thereafter are ignored for that execution. The updates will be reflected from the next execution. Rules defined in the datascan must be empty if this field is enabled.
| Parameter | |
|---|---|
| Name | Description |
var |
bool
|
| Returns | |
|---|---|
| Type | Description |
$this |
|
getFilter
Optional. Filter for selectively running a subset of rules. You can filter the request by the name or attribute key-value pairs defined on the rule.
If not specified, all rules are run. The filter is applicable to both, the rules retrieved from catalog and explicitly defined rules in the scan. Please see filter syntax for more details.
| Returns | |
|---|---|
| Type | Description |
string |
|
setFilter
Optional. Filter for selectively running a subset of rules. You can filter the request by the name or attribute key-value pairs defined on the rule.
If not specified, all rules are run. The filter is applicable to both, the rules retrieved from catalog and explicitly defined rules in the scan. Please see filter syntax for more details.
| Parameter | |
|---|---|
| Name | Description |
var |
string
|
| Returns | |
|---|---|
| Type | Description |
$this |
|