Ataccama ONE Agentic Cloud Portal ONE Desktop ONE DQ&C ONE MDM ONE RDM ONE Runtime Server
latest (17.0.0) 16.3.0 16.2.0 16.1.0 16.0.0 15.4.0 15.3.0 15.2.0 15.1.0 14.5.x 13.9.x 13.8.x 13.7.x 13.6.x 13.5.0 13.4.x 13.3.x 13.2.x
User Community Service Desk Downloads

Install Data Quality Gates for Snowflake UDFs

Follow this guide to deploy Data Quality Gates as Snowflake user-defined functions (UDFs). Once the installation is finished, each deployed DQ firewall is available in Snowflake as a UDF named atc_dq_<firewall-name>.

For synchronization after the initial rollout, see Synchronize Snowflake UDFs with ONE.

Supported ONE versions

DQ Gates are supported in the following versions of Ataccama ONE:

  • Ataccama ONE 16.3.0 and later.

Before you begin

Prerequisites

Make sure you’ve reviewed prerequisites and limitations before continuing.

What you’ll do

To install DQ Gates for Snowflake, you prepare your local environment, Ataccama ONE, and Snowflake, then run the provided Jupyter notebook that deploys your DQ firewalls as Snowflake UDFs.

The installation process includes:

  1. Preparing your local environment, Ataccama ONE, and Snowflake.

  2. Downloading the DQ Gates package.

  3. Installing dependencies.

  4. Uploading the Ataccama ONE Python libraries to Snowflake.

  5. Configuring the .env file.

  6. Running the installation notebook to deploy the firewalls as UDFs.

The preparation steps (1-5) ensure both environments are ready. The notebook then guides you through fetching firewalls from ONE and converting them to UDFs in Snowflake.

What you’ll need

Before starting installation, ensure that you have:

DQ Gates package

Contact your Customer Success Manager to receive the package.

Ataccama ONE

An account with access to the DQ firewalls that you want to deploy, and access to Keycloak in your ONE instance if you plan to use OIDC.

Snowflake

A database and warehouse, either existing or with permissions to create them, plus permissions to create objects within the database.

Required knowledge

You should be familiar with:

Python

Basic Python development and package management.

Snowflake

Snowflake authentication options, SQL operations, and basic validation of Snowflake objects.

Prepare Snowflake

Prepare Snowflake according to the requirements in Requirements overview.

The following sections also include detailed instructions for:

If Snowflake preparation is handled by another person, send them these requirements and ask them to provide you the results.

Requirements overview

To deploy DQ firewalls to Snowflake, you need the following:

  1. An active Snowflake account.

  2. A dedicated Snowflake warehouse, database, schema, and stage. These objects are where the UDFs are created. We recommend using a dedicated schema for the Ataccama UDF functions.

  3. A user account with access to those objects and the CREATE FUNCTION privilege in the target schema.

  4. Snowflake credentials and connection details for the .env file.

Snowflake credentials and connection details in the .env file
SNOWFLAKE_ACCOUNT="account"
SNOWFLAKE_USER="username"
SNOWFLAKE_PRIVATE_KEY_FILE="path/to/key/file"
# Alternatively, use password authentication
# SNOWFLAKE_PASSWORD="psw"
SNOWFLAKE_ROLE="role"
SNOWFLAKE_WAREHOUSE="WAREHOUSE"
SNOWFLAKE_DATABASE="DB"
SNOWFLAKE_SCHEMA="SCHEMA"
SNOWFLAKE_STAGE="@STAGE"
How to create Snowflake objects

To create the required Snowflake objects:

  1. Make sure that you have the CREATE WAREHOUSE, CREATE DATABASE, CREATE SCHEMA, and CREATE STAGE privileges. If you do not have these privileges, contact your Snowflake administrator.

  2. Create the required objects according to the Snowflake documentation:

How to set up a Snowflake user account

To set up a Snowflake user:

  1. Create a Snowflake user according to your selected authentication method:

  2. Make sure that this user has access to the relevant objects and the CREATE FUNCTION privilege in the target schema.

How to find Snowflake connection details

To find the Snowflake connection details:

  1. In Snowflake, select your account in the lower-left corner and then select Connect a tool to Snowflake.

  2. In the Account Details dialog:

    • On the Account tab, copy Account Identifier (SNOWFLAKE_ACCOUNT), User Name (SNOWFLAKE_USER), and Role (SNOWFLAKE_ROLE).

    • On the Config File tab, select your warehouse, database, and schema, then copy warehouse (SNOWFLAKE_WAREHOUSE), database (SNOWFLAKE_DATABASE), and schema (SNOWFLAKE_SCHEMA).

  3. For SNOWFLAKE_STAGE, use the name of your dedicated Snowflake stage. You can also find it under Data > Databases after you select your database and schema.

Security considerations

Lookups in UDFs are currently visible to everyone with access to the UDF. Take this into consideration when you deploy DQ firewalls that contain lookup tables with sensitive information.

We recommend:

  • Reviewing lookup data for sensitive information before deployment.

  • Applying appropriate permissions to the schemas that contain the DQ Gates UDFs.

Install DQ Gates

Follow these steps to install DQ Gates.

Download the DQ Gates package

Extract the ataccama-one-snowflake-<version>.zip package that you received to your local environment.

The package contains the components needed to set up and use the Ataccama ONE SDK for Snowflake:

  • ataccama_one-<version>-py3-none-any.whl

  • ataccama_one_expressions-<version>-py3-none-any.whl

  • firewall_to_snowflake_udf.ipynb

  • requirements.txt

  • .env.snowflake.example

  • ataccama-one-snowflake.py

  • README.md

Install dependencies

Install the dependencies needed to deploy DQ firewalls to Snowflake using one of the following options:

  • Install all dependencies at once using requirements.txt.

    pip install -r requirements.txt

  • Install the dependencies directly.

    pip install ataccama_one-0.0.0-py3-none-any.whl snowflake-snowpark-python python-dotenv jupyter

Dependencies needed to deploy DQ firewalls to Snowflake:

  • ataccama_one-<version>-py3-none-any.whl (from the downloaded package): Python SDK for Ataccama ONE.

  • ataccama_one_expressions-<version>-py3-none-any.whl (from the downloaded package): Python SDK for Ataccama ONE Expressions.

  • snowflake-snowpark-python: Python SDK for Snowflake Snowpark. Used to interact with Snowflake and perform data processing within Snowflake using Python.

    Why is Snowpark required?

    DQ Gates for Snowflake use Snowpark to integrate with Snowflake. Snowpark handles authentication and connection to Snowflake. It is also required for deploying UDFs to your Snowflake account.

    For alternative authentication approaches, see Snowpark documentation.

  • python-dotenv: Python package used for loading environment variables from the .env file.

  • Jupyter: Used for running the provided Jupyter Notebook that guides you through the installation process. If you prefer to run the code manually, Jupyter is not required.

Upload Ataccama ONE Python libraries to Snowflake

Upload the Ataccama ONE Python wheel files to the Snowflake stage so that the deployed UDFs can access those libraries inside Snowflake.

You can use one of the following methods:

Upload the following wheel files from the package:

  • ataccama_one-<version>-py3-none-any.whl

  • ataccama_one_expressions-<version>-py3-none-any.whl

Configure the .env file

Create an .env file in your project directory using .env.snowflake.example as a template. This file stores the environment variables needed to connect your local environment to Ataccama ONE and Snowflake during deployment.

env file template
ATACCAMA_INSTANCE_URL="https://myenv.ataccama.one/"
ATACCAMA_PLATFORM_VERSION="16.1"
ATACCAMA_CLIENT_ID="myclient"
ATACCAMA_CLIENT_SECRET="secret"
ATACCAMA_KEYCLOAK_HOST="keycloak-worker.ataccama.cloud"
ATACCAMA_KEYCLOAK_REALM="myrealm"
# Alternatively, use Basic Auth
# ATACCAMA_CLIENT_USERNAME="ataccama username"
# ATACCAMA_CLIENT_PASSWORD="password"

SNOWFLAKE_ACCOUNT="account"
SNOWFLAKE_USER="username"
SNOWFLAKE_PRIVATE_KEY_FILE="path/to/key/file"
# Alternatively, use password authentication
# SNOWFLAKE_PASSWORD="psw"
SNOWFLAKE_ROLE="role"
SNOWFLAKE_WAREHOUSE="WAREHOUSE"
SNOWFLAKE_DATABASE="DB"
SNOWFLAKE_SCHEMA="SCHEMA"
SNOWFLAKE_STAGE="@STAGE"

Ataccama ONE connection

  1. Set the following connection details:

    • ATACCAMA_INSTANCE_URL

    • ATACCAMA_PLATFORM_VERSION

  2. Select one of the following authentication methods:

    • OpenID Connect (OIDC). Configure ATACCAMA_CLIENT_ID, ATACCAMA_CLIENT_SECRET, ATACCAMA_KEYCLOAK_HOST, and ATACCAMA_KEYCLOAK_REALM.

    • Basic authentication. Configure ATACCAMA_CLIENT_USERNAME and ATACCAMA_CLIENT_PASSWORD, and comment out the OIDC variables.

Snowflake connection

  1. Set the following connection details:

    • SNOWFLAKE_ACCOUNT

    • SNOWFLAKE_USER

    • SNOWFLAKE_ROLE

    • SNOWFLAKE_WAREHOUSE

    • SNOWFLAKE_DATABASE

    • SNOWFLAKE_SCHEMA

    • SNOWFLAKE_STAGE

  2. Select one of the following authentication methods:

    • Key-pair authentication. Recommended. Configure SNOWFLAKE_PRIVATE_KEY_FILE.

    • Basic authentication. Configure SNOWFLAKE_PASSWORD, and comment out SNOWFLAKE_PRIVATE_KEY_FILE.

Run installation

Run Jupyter Notebook and follow the instructions in it to register DQ firewalls as Snowflake UDFs.

The notebook guides you through these steps:

  1. Set up your local Python environment and connect to Ataccama ONE.

  2. Fetch and download DQ firewalls from Ataccama ONE.

  3. Connect to Snowflake.

  4. Upload the firewalls to Snowflake and convert them to Snowflake UDFs.

For common endpoint configuration and firewall selection options, see Custom URL resolution for self-managed deployments and Firewall filtering.

Conflict strategies

Available from DQ Gates version 1.1.0.

Control how existing UDFs with the same name, signature, or firewall ID are handled during deployment.

  • Using built-in strategies. Pass a strategy name to register_firewall_in_snowflake:

    • skip_on_conflict

    • override_on_conflict

    • keep_both_or_skip

    • keep_both_or_override

      Built-in strategy example
      from ataccama_one_snowflake import register_firewall_in_snowflake

register_firewall_in_snowflake(
    session=session,
    firewall_zip_file_path="my_firewall.zip",
    stage_location="@stage",
    udf_name="validate_customers",
    conflict_strategy="keep_both_or_override",
)

  • Using custom strategies. Supply a callable that returns a ConflictResolution for fine-grained control.

    The metadata used for conflict resolution is embedded in the Snowflake UDF comment. Avoid editing comment fields manually, as this breaks future conflict checks.

    You can supply your own additional metadata when registering a firewall — it will be merged into the stored comment without overwriting the fields that conflict strategies rely on.
    Custom strategy example
    from ataccama_one_snowflake import ConflictResolution, register_firewall_in_snowflake

def drop_old_firewalls(conflicts, candidate_name, metadata) -> ConflictResolution:
    drops = []
    for udf in conflicts:
        created = udf.get("created")
        if created and created < metadata["threshold"]:
            drops.append(f"DROP FUNCTION IF EXISTS {udf['udf_name']}({udf['arg_sig']})")
        else:
            return ConflictResolution(
                abort=True,
                reasons=["Recent firewall already exists"],
            )
    return ConflictResolution(pre_sql=drops)

register_firewall_in_snowflake(
    session=session,
    firewall_zip_file_path="my_firewall.zip",
    stage_location="@stage",
    udf_name="validate_customers",
    conflict_strategy=drop_old_firewalls,
)
If your custom strategy needs additional information (for example, creation timestamps or owner details), query INFORMATION_SCHEMA.FUNCTIONS before calling register_firewall_in_snowflake and pass the enriched data into your resolver.

Verify the deployment

After the notebook finishes, validate that the UDFs exist in the target schema and continue with Evaluate Data Quality in Snowflake with UDFs.

Was this page useful?