If you can't find the product or version you're looking for, visit support.ataccama.com/downloads

Constraints Configuration

Constraints are meant to be used as helpers for routing jobs to specific Data Processing Engines (DPEs).

By default, each DPE has access to every available data source and file system and can be employed in any environment and for any type of jobs. Using constraints, you can restrict or override the individual capabilities of a DPE instance, which are usually derived from the installed plugins and other settings.

As the capabilities of each DPE are taken into account when Data Processing Module (DPM) manages requests and assigns jobs, this can ensure that DPE has the resources to successfully handle incoming requests from DPM.

The value of a constraining property is typically a comma-separated list of one or more string values referring to capabilities, with a straight double quote (") used as a quoting character and a backslash (\) used as an escape character. When using capabilities with different value types as elements of an enumeration, start the property value with the type specifier enclosed in square brackets ([<type_specifier>]), for example, [pattern] or [string].

You can have multiple types of elements in an enumeration, but when using properties to set them, the same type is assumed for all enumeration elements, as defined by the type specifier. The following type specifiers are allowed: [string], [pattern], [long], [integer], [double], [float], [boolean]. Intervals, custom types, and intervals of custom types are currently not supported.

If a constraint is defined as an environment variable, the same value type must be used for all elements of an enumeration. Nesting enumerations is not allowed. Furthermore, constraints used as environment properties accept only primitive data types that are supported by the previously mentioned type specifiers of property values.

We recommend configuring constraints using DPM Admin Console. In self-managed, on-premise deployment, the initial values for these properties can also be provided in the dpe/etc/application.properties file, but any editing should be done only through DPM Admin Console.

Property Data type Description

constraints.com.ataccama.dpe.plugin.dataconnect.provider.type

String

Specifies the types of data sources that DPE can access. By default, each DPE can access any available data source.

You can view the available values for this property in DPM Admin Console > Engines > [your DPE instance]. The values are located in Constraints, in the com.ataccama.dpe.plugin.dataconnect.provider.type array.

To configure the property using DPM Admin Console, go to DPM Admin Console > Configuration > DPE Configurations. In Capability Overrides, find and select Data-connect Provider Type.

To add a new value, select Add Capability and enter the data source name in Value. The data type should be String. Select Confirm to finish adding the data source. Repeat this for each source you want to configure.

Constraints configuration in DPM Admin Console

Example (in dpe/etc/application.properties): FILESYSTEM,GDRIVE,RDM,POSTGRESQL,ORACLE.

constraints.com.ataccama.dpe.plugin.executor.provider.type

String

Defines the types of jobs that are directed to this DPE.

You can view the available values for this properties in DPM Admin Console > Engines > [your DPE instance]. The values are located in Constraints, in the com.ataccama.dpe.plugin.executor.provider.type array.

To configure the property using DPM Admin Console, go to DPM Admin Console > Configuration > DPE Configurations. In Capability Overrides, find and select Executor Provider Type.

To add a new value, select Add Capability and enter the job type in Value. The data type should be String. Select Confirm. Repeat this for each job type you want to configure.

Example (in dpe/etc/application.properties): METADATA-IMPORT,DQC,DQM,PUSHDOWN-PROFILING.

constraints.com.ataccama.dpe.plugin.dataconnect.local.fs.datasource.mount

String

Determines which file systems this DPE can access.

Default value: default.

constraints.com.ataccama.dpe.plugin.dataconnect.jdbc.connectionstring

String

Specifies to which data sources DPE can connect.

The allowed connection strings are provided here in the form of a pattern. You can also provide full connection strings for each data source.

We strongly recommend not using patterns that contain potentially sensitive values or parameters.

Default value: [pattern]jdbc:oracle:.,jdbc:postgresql:.,jdbc:mysql:.,jdbc:sqlserver:.,jdbc:h2:.*.

constraints.com.ataccama.dpe.plugin.dqc.datasource.url

String

By default, DQC data source URLs do not exclude ONE jobs using their own drivers. This property restricts that behavior in order to allow using only registered DataConnect or Metastore drivers and can be used for routing jobs to a specific DPE. The value can be also set to a custom expression instead.

Default value: [pattern].*.

constraints.com.ataccama.dpe.plugin.dataconnect.adls.container

String

Determines the storage account name and the container name of the ADLS connection that should be processed by this DPE.

The value of this constraint should be structured as follows: container_name@storage_account_name.

constraints.com.ataccama.dpe.plugin.dataconnect.managed.identity

String

Defines the managed identity ID in DPE.

constraints.com.ataccama.dpe.plugin.dataconnect.aws.instance.iam

Boolean

Allows authentication to Amazon S3 server with IAM credentials specified in Metadata Management Module (MMM). DPE has to be deployed in an EC2 instance for this property to be set.

If set to true, this DPE is ready to accept requests with IAM credentials.

Default value: [boolean]false.

Was this page useful?