Cloud Portal ONE Desktop ONE DQ&C ONE MDM ONE RDM ONE Runtime Server
User Community Service Desk Downloads

Enable Snowflake Pushdown for DQ Evaluation on Source

How to enable Snowflake pushdown processing for DQ evaluation

To enable pushdown processing for DQ evaluation on your datasource you need to edit the Snowflake connection details.

  1. In Catalog > Sources, select your Snowflake source and edit the connection details, or create a new connection.

    Add new connection

    1. In Catalog > Sources, select your Snowflake source.

      If you need to create a new source, see Create a source in Snowflake Connection.

    2. Select Add connection.

    3. In Connection type, select Snowflake.

    4. Provide the following:

      • Name: A meaningful name for your connection. This is used to indicate the location of catalog items.

      • Description (Optional): A short description of the connection.

      • JDBC: A JDBC connection string pointing to the IP address or the URL where the data source can be reached.

        The connection string should take the form:

        jdbc:snowflake://<account_identifier>.snowflakecomputing.com/?<connection_params>

        For more information, see the official Snowflake documentation.

    5. Proceed to step 2.

    Edit existing connection

    1. In Catalog > Sources, select your Snowflake source.

    2. Select the Connections tab, and click the connection name.

    3. Select the three dots menu for the connection you want, and then select Edit.

      edit source connection

    4. Proceed to step 2.

  2. In the Pushdown processing section, enable Evaluate Data Quality using pushdown processing.

    Ataccama uses an allocated user stage @~/ataccama_cache/` in the Snowflake user account used in the connection as the default location for storing Ataccama lookups. You can change the designated stage in the advanced settings if needed.

    At this point no further action is required, unless you want to use something other than the default stage to store Ataccama lookups.

  3. Save your changes to proceed, or see custom named stage if you want to specify a different stage.

Use custom named stage

You can use a named stage instead of the default user stage.

For more information about stage types, see Snowflake’s guide to the types of internal stages.

It is recommended to use a dedicated stage to store Ataccama lookups. If necessary, create a new stage according to the Snowflake’s instructions.

Once your named stage is ready in Snowflake, return to the connection settings in ONE and select Advanced pushdown settings.

Enter the path to the stage in Stage location and Save your changes.

Why does Ataccama use my Snowflake stage?

Any lookup data used in rules must be stored in the connected Snowflake account, available for processing.

Ataccama uses PUT, LIST and REMOVE commands to manage lookup data in Snowflake. Lookups are updated when you DQ evaluation and profiling jobs, and are removed automatically if they have been inactive for more than 30 days. Filenames in Snowflake take the format: ${this.stageLocation.stageLocation}${namePrefix}${dayHex}_${timeStr}_lookup_${contentHash}_$sanitizedNameStr.csv

Was this page useful?