Ataccama ONE Agentic ONE Desktop ONE DQ&C ONE MDM ONE RDM ONE Runtime Server
Downloads

Generic Connector

This article describes how to connect to a database that has no native connector in Ataccama ONE, using the Generic connector and your own JDBC driver.

With the Generic connector, you package a JDBC driver and its configuration as an OCI artifact, publish the artifact to a container registry your edge instance can reach, and deploy it to the edge instance directly when setting up the connection. Processing jobs on the edge instance then load the driver at runtime and work with the source as with any other JDBC database.

Availability

Data processing & catalog Edge processing Lineage Exceptions

✔

✔

✗

Edge processing: Required. The Generic connector is available only on connections processed by an edge instance.

Pushdown processing is not supported.

Prerequisites

  • Review how sources and connections work.

  • Create a source to add this connection to.

  • Register an edge instance on version 2026.08.04.0 or later in your environment. See Edge Processing. Earlier versions do not support deploying drivers from your own registry.

  • Obtain the JDBC driver JAR file from your database vendor and make sure your license permits this use.

  • Prepare an OCI-compliant container registry that the edge instance can reach over the network, with permission to push artifacts to it.

  • Install the ORAS CLI version 1.2 or later on the machine you publish the driver from.

Package the driver bundle

A driver bundle is a directory containing:

  • One or more JDBC driver JAR files with the .jar extension. The base names don’t matter; all JAR files in the bundle are loaded together.

  • Exactly one .properties file with the driver configuration, named <name>.properties.

All files must be non-empty and placed directly in the directory, without subdirectories or symbolic links.

<name> must be the last segment of the repository path you publish the bundle to, in lowercase. For example, for a bundle published as 123456789012.dkr.ecr.eu-central-1.amazonaws.com/jdbc-drivers/mydb:1.0.0, the configuration file must be named mydb.properties.

In addition, every key in the configuration file must start with ataccama.one.driver.mydb.. The connector derives both the configuration file name and the key prefix from the Artifact URL of the connection, so a mismatch means the driver fails to load even though the deployment succeeded.

Write the driver configuration

The .properties file tells the connector how to load the driver and which SQL to generate. Every key starts with the prefix ataccama.one.driver.<name>.; the following tables omit the prefix for brevity.

Keys the connector doesn’t recognize are ignored.

Contents of mydb.properties
ataccama.one.driver.mydb.driver-identifier=MYDB
ataccama.one.driver.mydb.driver-class-name=com.example.jdbc.Driver
ataccama.one.driver.mydb.schema-strategy=SCHEMA_AWARE
ataccama.one.driver.mydb.quotation-mark="
ataccama.one.driver.mydb.full-select-query-pattern=SELECT {columns} FROM {table}
ataccama.one.driver.mydb.path-preview-query-pattern=SELECT {columns} FROM {table} LIMIT {previewLimit}
ataccama.one.driver.mydb.dsl-query-preview-query-pattern=SELECT * FROM ({dslQuery}) dslQuery LIMIT {previewLimit}
ataccama.one.driver.mydb.dsl-query-import-metadata-query-pattern=SELECT * FROM ({dslQuery}) dslQuery LIMIT 0
ataccama.one.driver.mydb.row-count-query-pattern=SELECT COUNT(*) FROM {table}
ataccama.one.driver.mydb.sampling-query-pattern=SELECT {columns} FROM {table} WHERE RANDOM() < {percentageLimit} LIMIT {limit}

Required keys

If a required key is missing, driver deployment still succeeds, but the connection fails when it is first tested or used.

The query pattern keys are SQL templates the connector substitutes into at query time; write them in the SQL dialect of your database.

Placeholders in curly braces are interpolated when the query runs:

  • {columns}, {table}, {simpleTable}, {schema}, and {database} identify what to read.

  • {previewLimit} is available in the two preview patterns.

  • {limit}, {percentageLimit}, and {percentage100Limit} are used in the sampling pattern.

  • {dslQuery} is used in the two dsl-query-* patterns, which don’t receive {columns} or {table}.

A placeholder used in a pattern that doesn’t support it is left in the SQL unchanged.

Key Description

driver-class-name

The fully qualified class name of the JDBC driver. If you are not sure, check the file META-INF/services/java.sql.Driver inside the driver JAR.

driver-identifier

A symbolic identifier of the driver, by convention in UPPER_SNAKE_CASE. The value also selects the SQL dialect used for generated queries, by case-insensitive substring match: ORACLE, POSTGRES, MSSQL, AZURE, FABRIC, MYSQL, MARIADB, HANA, or the exact values BQ, SNOWFLAKE, REDSHIFT, TERADATA.

Pick a value that matches your database’s dialect, or one that contains none of these tokens to get the generic default.

full-select-query-pattern

Full read of a table.

Example: SELECT {columns} FROM {table}.

path-preview-query-pattern

Bounded preview of a table.

Example: SELECT {columns} FROM {table} LIMIT {previewLimit}.

dsl-query-preview-query-pattern

Preview of a user-supplied SQL query.

Example: SELECT * FROM ({dslQuery}) dslQuery LIMIT {previewLimit}.

dsl-query-import-metadata-query-pattern

Metadata probe of a user-supplied SQL query: it must expose the result schema while returning no rows.

Example: SELECT * FROM ({dslQuery}) dslQuery LIMIT 0.

row-count-query-pattern

Row count of a table.

Example: SELECT COUNT(*) FROM {table}.

sampling-query-pattern

Random or bounded sample used by profiling. If the database has no RANDOM() function or TABLESAMPLE clause, fall back to a plain SELECT {columns} FROM {table} LIMIT {limit}.

Optional keys

Key Default Description

schema-strategy

SCHEMA_AWARE

How the connector browses the source:

  • SCHEMA_AWARE: Database > schema > table, for drivers that report a stable relational schema through JDBC metadata.

  • SCHEMALESS: Database > table, for document-flavored sources without a fixed schema.

quotation-mark

None

The identifier quote character of the SQL dialect, typically a double quotation mark (") or a backtick (`). Without quotation-mark, identifiers are not quoted at all, so always set it unless the database forbids quoting.

allowed-table-types

TABLE,VIEW,PARTITIONED TABLE

Comma-separated JDBC table types eligible for browsing and metadata import.

disallowed-indexes-table-types

None

Comma-separated table types to skip when reading index metadata, for example, SYNONYM or VIEW for databases that cannot index views.

disallowed-indexes-schema-names

None

Comma-separated schema names to skip when reading index metadata.

schema-include-pattern, schema-exclude-pattern

None

Regular expression filters applied to schema names during discovery.

table-include-pattern, table-exclude-pattern

None

Regular expression filters applied to table names during discovery.

bulk-import-table-count-threshold

None

Above this number of tables, metadata import switches to the bulk import path.

location-query

The driver’s JDBC catalog

An SQL query that returns a single value, used to name the root location of the connection in the catalog.

Catalog items are matched by their location, not by their connection: when two connections resolve to the same root location name, they share catalog items, and every metadata import reassigns the shared items to the connection that ran it. To prevent this with drivers that report the same catalog name for every connection, set the query to a value specific to the database instance, such as SELECT current_database().

The query runs every time the connection is established, so use a simple query the connection user can always run. If the query fails, the connection itself fails, not just the naming.

row-count-timeout

PT119M

Timeout for row count queries, as an ISO-8601 duration.

supports-table-filter

true

Set to false for drivers that ignore the table name pattern of the JDBC metadata API; the connector then filters tables itself.

Create table templates

To export data to this source, the configuration must also define how tables are created, under the create-table. sub-prefix:

Create table keys in mydb.properties
ataccama.one.driver.mydb.create-table.table-pattern=CREATE TABLE {table} ({columns})
ataccama.one.driver.mydb.create-table.column-pattern={column} {type}
ataccama.one.driver.mydb.create-table.fragment-separator=,\n
ataccama.one.driver.mydb.create-table.type-mappings.STRING=varchar({columnSize = 128})
ataccama.one.driver.mydb.create-table.type-mappings.STRING.lob=text
ataccama.one.driver.mydb.create-table.type-mappings.STRING.lob.when=columnSize > 2000
ataccama.one.driver.mydb.create-table.type-mappings.INTEGER=integer
ataccama.one.driver.mydb.create-table.type-mappings.LONG=bigint
ataccama.one.driver.mydb.create-table.type-mappings.FLOAT=numeric({columnSize = 32},{fractionalDigits = 8})
ataccama.one.driver.mydb.create-table.type-mappings.BOOLEAN=boolean
ataccama.one.driver.mydb.create-table.type-mappings.DATE=date
ataccama.one.driver.mydb.create-table.type-mappings.DATETIME=timestamp

To export to the source, you need to set create-table.table-pattern; otherwise, exporting isn’t available, though everything else works as usual. If you do set it, column-pattern and fragment-separator are required as well: leaving them out prevents the whole connection from working, not just the export.

The type-mappings.<TYPE> keys map the platform data types (STRING, INTEGER, LONG, FLOAT, BOOLEAN, DATE, DATETIME) onto the SQL types of your database. You can use two placeholders inside a mapping value: {columnSize} and {fractionalDigits}, replaced with the size and precision of the column being created. The form {columnSize = 128} additionally sets the default used when the column doesn’t define the value; no other fields can be used inside mapping values.

An optional named variant selects an alternative SQL type; every variant must carry a .when condition.

Conditions can compare the string fields jdbcColumnType and nativeColumnType (equality only) and the numeric fields columnSize, fractionalDigits, and jdbcColumnTypeNumber, combined with && or ||. Keep the conditions of a type’s variants mutually exclusive because overlapping conditions have no defined precedence.

Publish the bundle to a registry

Start by authenticating the ORAS CLI to your registry. For a registry with username and password authentication:

Authenticate to a registry with a username and password
oras login <REGISTRY_HOST>

For Amazon ECR, exchange your AWS identity for a temporary registry token (valid for 12 hours):

Authenticate to Amazon ECR
aws ecr get-login-password --region <REGION> |
    oras login --username AWS --password-stdin <ACCOUNT_ID>.dkr.ecr.<REGION>.amazonaws.com

Replace <ACCOUNT_ID> and <REGION> with the AWS account and region of the ECR registry.

Then push the bundle as an OCI artifact. Run the command from inside the bundle directory so that the artifact references bare file names:

Push the driver bundle
cd <BUNDLE_DIRECTORY>
oras push <REGISTRY_HOST>/<REPOSITORY>:<TAG> \
    --artifact-type application/vnd.ataccama.driver-bundle.v1+json \
    <DRIVER_JAR_FILE>:application/vnd.ataccama.jdbc.jar \
    <NAME>.properties:application/vnd.ataccama.driver.config

Replace the placeholders as follows. Example values are for a PostgreSQL driver bundle pushed to Amazon ECR.

  • <BUNDLE_DIRECTORY>: The driver bundle directory.

    Example: postgresql/ (containing postgresql-42.7.10.jar and postgresql.properties).

  • <DRIVER_JAR_FILE>: The JAR file name. Repeat the argument for each JAR in the bundle.

    Example: postgresql-42.7.10.jar.

  • <NAME>: The bundle name. See Package the driver bundle.

    Example: postgresql.

  • <REGISTRY_HOST>: The host of your registry.

    Example: 123456789012.dkr.ecr.us-east-1.amazonaws.com (an Amazon ECR registry in account 123456789012 and region us-east-1).

  • <REPOSITORY>: The repository path within the registry.

    Example: edge-drivers/postgresql (created beforehand, see Use Amazon ECR).

  • <TAG>: The version tag you choose for the bundle, see Version the bundle.

    Example: 42.7.10, matching the driver version.

With these example values, the push becomes:

cd postgresql
oras push 123456789012.dkr.ecr.us-east-1.amazonaws.com/edge-drivers/postgresql:42.7.10 \
    --artifact-type application/vnd.ataccama.driver-bundle.v1+json \
    postgresql-42.7.10.jar:application/vnd.ataccama.jdbc.jar \
    postgresql.properties:application/vnd.ataccama.driver.config

The Artifact URL of this bundle is then 123456789012.dkr.ecr.us-east-1.amazonaws.com/edge-drivers/postgresql:42.7.10.

The artifact must follow this structure; otherwise, the edge instance rejects it during deployment:

  • The artifact type is exactly application/vnd.ataccama.driver-bundle.v1+json.

  • Each driver JAR is one layer with the media type application/vnd.ataccama.jdbc.jar. At least one JAR layer is required.

  • The configuration file is exactly one layer with the media type application/vnd.ataccama.driver.config.

  • No other layers are allowed.

The full artifact reference, which you later provide as the Artifact URL of the connection, has the form <registry_host>/<repository>:<tag>. The registry host and an explicit tag are both mandatory; referencing by digest (@sha256:…​) is not supported. Don’t include a URL scheme such as https://.

The last segment of the repository path also names the bundle configuration (see Package the driver bundle).

Version the bundle

Use the tag as the version of the bundle. We recommend using the driver version itself (for example, 42.7.12), with a build suffix when you republish the same driver JAR with a changed configuration (42.7.12-1, 42.7.12-2).

The edge instance deploys each tag at most once. If you push different content under a tag that an edge instance has already deployed, that edge instance keeps the original version.

Treat tags as immutable and publish every change (whether a new driver JAR or a configuration edit) under a new tag, then update the Artifact URL of the connection. For the same reason, avoid a floating latest tag: the edge instance would keep whatever latest pointed to at first deployment and never pick up updates.

Use Amazon ECR

For edge instances running in AWS, Amazon ECR is the most direct option.

The edge instance authenticates with its own IAM role, so no registry credentials are configured in Ataccama ONE. When the Artifact URL points to a standard Amazon ECR endpoint (<account_id>.dkr.ecr.<region>.amazonaws.com), the Artifact URL user name reference and Artifact URL password reference fields are automatically grayed out.

Because the pull relies on the role’s permissions, only repositories in the same AWS account as the edge instance are supported. Create the ECR repository in that account before pushing: Amazon ECR does not create repositories on push. The repository can be in any region.

With the default setup, no additional IAM configuration is required on your side: the permissions for the pull are part of the edge instance infrastructure. If your organization restricts Amazon ECR with explicit-deny repository or registry policies, additionally allow the edge instance’s deploy driver job task role to pull from the driver repository.

Driver distribution from Amazon ECR is configured independently of the registry that serves the edge instance’s own container images, but the network path must still allow it: the edge instance needs access to the regional Amazon ECR endpoints and to Amazon S3, which serves the layer downloads.

If you also serve the edge instance’s container images from Amazon ECR, see Set Up Amazon ECR for Edge Container Images.

Use other registries

For any registry other than Amazon ECR, the edge instance authenticates with a username and password sourced from your secret management service. Anonymous (unauthenticated) registries are not supported.

To use a non-Amazon ECR registry:

  1. Store the registry username and the registry password as two secrets in AWS Secrets Manager that the edge instance can access. Store each value as a plaintext secret, not as a JSON key/value secret: the whole secret value is used verbatim as the username or password.

  2. Add AWS Secrets Manager as a secret management service if you haven’t already. See Secret Management Service and Set Up AWS Secrets Manager Access for Edge.

  3. When adding the connection, select the secret management service and provide the names of the two secrets in Artifact URL user name reference and Artifact URL password reference.

Add a connection

  1. Go to [your source] > Connections and select Add Connection.

  2. In Connection type, select Generic connector.

  3. Fill in the following:

    • Name: A meaningful name for your connection. Used to indicate the location of catalog items.

    • Description (Optional): A short description of the connection.

    • Edge instance: The edge instance that runs this connection’s jobs. See Edge Processing.

      The driver is deployed to this instance, and the edge instance cannot be changed after the connection is created. To change to a different edge instance, you need to create a new connection instead. * To allow exporting data processed on the edge to Reference Data, select *Allow Edge export. See Allow edge export.

    • Secret management service (Optional): The secret management service used to retrieve secret values at runtime. See Secret Management Service.

      Required if the registry needs username and password authentication or if you source driver property values from secrets. For registry credentials, the service must be of the AWS Secrets Manager type (see Use other registries).

    • JDBC: The JDBC connection string, in the format your driver expects.

    • Artifact URL: The full reference of the published driver bundle, <registry_host>/<repository>:<tag>. See Publish the bundle to a registry.

    • Artifact URL user name reference and Artifact URL password reference (Optional): The names of the secrets holding the registry username and password. Fill in both fields, or neither.

      For Amazon ECR, leave both empty.

  4. Add the driver properties your driver needs, typically including the database credentials. See Add driver properties.

    You can also return to this step after you save the connection, but until the credentials are in place, the connection test fails.

  5. Select Deploy driver. The edge instance pulls the driver bundle from the registry, verifies it, and stores it locally.

    Deploying a driver typically takes under two minutes; if it takes longer, the deployment failed. See Troubleshooting.

  6. Wait until the deployment status updates to Driver deployed successfully and then save and publish the connection.

    The connection cannot be saved until the driver is deployed successfully. See also Deploy a new driver version.

Add driver properties

Driver properties are JDBC connection properties passed to the driver together with the JDBC connection string.

The property names are driver-specific: check the driver vendor’s documentation for what the driver requires or supports (for example, user, password, sslMode, or connectTimeout). This is typically also where you provide the database credentials.

  1. Select Add driver property.

  2. Fill in Property name.

  3. In Driver property type, select one of the following:

    • Value: A literal value entered in the form.

    • File: A file uploaded from disk, up to 5 MB. Use for drivers that take file-based credentials or trust material, such as a keystore, a certificate, or a service account key.

      For File properties, the value field is labeled Content and the JSON path field Content JSON path.

  4. For sensitive values such as passwords, select Secured so the value is masked.

    Secured cannot be changed after the property is saved, and Use secret management service turns it on automatically.

  5. To source the value from your secret management service instead of entering it directly, select Use secret management service on the property. This option is available only when the connection has a secret management service selected.

  6. Enter the name of the secret as the value.

    To pick a single field from a JSON-valued secret, fill in Value JSON path.

  7. After adding the properties, test the connection to verify that the driver loads and the credentials work. If the test fails even though the driver deployed successfully, see Troubleshooting.

Deploy a new driver version

The edge instance deploys each tag at most once and ignores content changes pushed under an already deployed tag.

To update the driver JAR or its configuration, publish the change under a new tag and point the connection to it:

  1. Publish the updated bundle under a new tag. See Version the bundle.

    This applies to configuration-only changes as well: republish the bundle with the edited .properties file under a new tag (for example, 42.7.12-2).

  2. In the connection settings, update Artifact URL to reference the new tag.

  3. Select Deploy driver.

    Wait until the status updates to Driver deployed successfully.

  4. Save the connection.

    Unlike when creating a connection, saving here is not blocked by the deployment status: if you save while the deployment is still running or after it has failed, jobs on this connection fail until a deployment succeeds.

  5. Test the connection to confirm the new driver version works.

Troubleshooting the Generic connector

Driver deployment fails

Verify that:

  • The Artifact URL includes an explicit tag. References without a tag are rejected.

  • The Artifact URL is correct and the tag exists in the registry.

  • The edge instance has network access to the registry host.

  • For Amazon ECR, the repository is in the same AWS account as the edge instance.

  • For other registries, both secret references are filled in and the secrets exist in your secret management service.

Connection test fails after a successful deployment

A successful deployment does not validate the driver configuration. Check that all required keys are present and that the configuration file name matches the repository path (see Package the driver bundle).

Was this page useful?