Install Data Quality Gates for Snowflake UDFs

Follow this guide to deploy Data Quality Gates as Snowflake user-defined functions (UDFs). Once the installation is finished, each deployed DQ firewall is available in Snowflake as a UDF named atc_dq_<firewall-name>.

For synchronization after the initial rollout, see Synchronize Snowflake UDFs with ONE.

Before you begin

Prerequisites

Make sure you’ve completed prerequisites and reviewed limitations.

What you’ll do

The installation process includes:

Preparing Snowflake.
Downloading the DQ Gates package.
Installing dependencies.
Uploading the Ataccama ONE Python libraries to Snowflake.
Configuring the Snowflake connection in the .env file.
Running the installation notebook to deploy the firewalls as UDFs.

The preparation steps (1-5) ensure your local environment and Snowflake are ready. The notebook then guides you through fetching firewalls from ONE and converting them to UDFs in Snowflake.

What you’ll need

DQ Gates package: Contact your Customer Success Manager to receive the package.
Snowflake: A database and warehouse, either existing or with permissions to create them, plus permissions to create objects within the database.

Required knowledge

You should be familiar with:

Python: Basic Python development and package management.
Snowflake: Snowflake authentication options, SQL operations, and basic validation of Snowflake objects.

Prepare Snowflake

Prepare Snowflake according to the requirements in Requirements overview.

The following sections also include detailed instructions for:

Creating Snowflake objects
Finding Snowflake connection details
Setting up a Snowflake user account

If Snowflake preparation is handled by another person, send them these requirements and ask them to provide you the results.

Requirements overview

To deploy DQ firewalls to Snowflake, you need the following:

An active Snowflake account.
A dedicated Snowflake warehouse, database, schema, and stage. These objects are where the UDFs are created. We recommend using a dedicated schema for the Ataccama UDFs.
A user account with access to those objects and the CREATE FUNCTION privilege in the target schema.
Snowflake credentials and connection details for the .env file (see [configure-snowflake-connection-in-the-.env-file]).

How to create Snowflake objects

To create the required Snowflake objects:

Make sure that you have the CREATE WAREHOUSE, CREATE DATABASE, CREATE SCHEMA, and CREATE STAGE privileges. If you do not have these privileges, contact your Snowflake administrator.
Create the required objects according to the Snowflake documentation:

How to set up a Snowflake user account

To set up a Snowflake user:

Create a Snowflake user according to your selected authentication method:
- Key-pair authentication. Recommended. For instructions, see Key-pair authentication and key-pair rotation.
- Basic authentication. This method is expected to be deprecated by Snowflake.
Make sure that this user has access to the relevant objects and the CREATE FUNCTION privilege in the target schema.

How to find Snowflake connection details

To find the Snowflake connection details:

In Snowflake, select your account in the lower-left corner and then select Connect a tool to Snowflake.
In the Account Details dialog:
- On the Account tab, copy Account Identifier (SNOWFLAKE_ACCOUNT), User Name (SNOWFLAKE_USER), and Role (SNOWFLAKE_ROLE).
- On the Config File tab, select your warehouse, database, and schema, then copy warehouse (SNOWFLAKE_WAREHOUSE), database (SNOWFLAKE_DATABASE), and schema (SNOWFLAKE_SCHEMA).
For SNOWFLAKE_STAGE, use the name of your dedicated Snowflake stage. You can also find it under Data > Databases after you select your database and schema.

Security considerations

Lookups in UDFs are currently visible to everyone with access to the UDF. Take this into consideration when you deploy DQ firewalls that contain lookup tables with sensitive information.

We recommend:

Reviewing lookup data for sensitive information before deployment.
Applying appropriate permissions to the schemas that contain the DQ Gates UDFs.

Install DQ Gates

Download the DQ Gates package

Extract the ataccama-one-snowflake-<version>.zip package that you received to your local environment.

The package contains the components needed to set up and use the Ataccama ONE SDK for Snowflake:

ataccama_one-<version>-py3-none-any.whl
ataccama_one_expressions-<version>-py3-none-any.whl
firewall_to_snowflake_udf.ipynb
requirements.txt
.env.snowflake.example
ataccama-one-snowflake.py
README.md

Install dependencies

Install the dependencies needed to deploy DQ firewalls to Snowflake using one of the following options:

Install all dependencies at once using requirements.txt.
```
pip install -r requirements.txt
```

Install the dependencies directly.

pip install ataccama_one-0.0.0-py3-none-any.whl snowflake-snowpark-python python-dotenv jupyter

Dependencies needed to deploy DQ firewalls to Snowflake:

ataccama_one-<version>-py3-none-any.whl (from the downloaded package): Python SDK for Ataccama ONE.
ataccama_one_expressions-<version>-py3-none-any.whl (from the downloaded package): Python SDK for Ataccama ONE Expressions.

snowflake-snowpark-python: Python SDK for Snowflake Snowpark. Used to interact with Snowflake and perform data processing within Snowflake using Python.

Why is Snowpark required?

DQ Gates for Snowflake use Snowpark to integrate with Snowflake. Snowpark handles authentication and connection to Snowflake. It is also required for deploying UDFs to your Snowflake account.

For alternative authentication approaches, see Snowpark documentation.

python-dotenv: Python package used for loading environment variables from the .env file.
Jupyter: Used for running the provided Jupyter Notebook that guides you through the installation process. If you prefer to run the code manually, Jupyter is not required.

Upload Ataccama ONE Python libraries to Snowflake

Upload the Ataccama ONE Python wheel files to the Snowflake stage so that the deployed UDFs can access those libraries inside Snowflake.

You can use one of the following methods:

Upload the files through the Snowflake web interface. See Upload files using the UI.
Upload the files using the PUT command in SnowSQL or another supported driver. See Upload files using SnowSQL.

Upload the following wheel files from the package:

ataccama_one-<version>-py3-none-any.whl
ataccama_one_expressions-<version>-py3-none-any.whl

Configure Snowflake connection in the .env file

In the .env file created during the prerequisites setup, add the Snowflake connection details. The Ataccama ONE variables should already be configured.

Complete .env file example

ATACCAMA_INSTANCE_URL="https://myenv.ataccama.one/"
ATACCAMA_PLATFORM_VERSION="saas"
ATACCAMA_CLIENT_ID="myclient"
ATACCAMA_CLIENT_SECRET="secret"
ATACCAMA_KEYCLOAK_HOST="myenv.ataccama.one"
ATACCAMA_KEYCLOAK_REALM="myrealm"

SNOWFLAKE_ACCOUNT="account"
SNOWFLAKE_USER="username"
SNOWFLAKE_PRIVATE_KEY_FILE="path/to/key/file"
# Alternatively, use password authentication
# SNOWFLAKE_PASSWORD="psw"
SNOWFLAKE_ROLE="role"
SNOWFLAKE_WAREHOUSE="WAREHOUSE"
SNOWFLAKE_DATABASE="DB"
SNOWFLAKE_SCHEMA="SCHEMA"
SNOWFLAKE_STAGE="@STAGE"

Set the following Snowflake connection details:
- SNOWFLAKE_ACCOUNT
- SNOWFLAKE_USER
- SNOWFLAKE_ROLE
- SNOWFLAKE_WAREHOUSE
- SNOWFLAKE_DATABASE
- SNOWFLAKE_SCHEMA
- SNOWFLAKE_STAGE
  
  For details on finding these values, see How to find Snowflake connection details.
Select one of the following authentication methods:
- Key-pair authentication. Recommended. Configure SNOWFLAKE_PRIVATE_KEY_FILE.
- Basic authentication. Configure SNOWFLAKE_PASSWORD, and comment out SNOWFLAKE_PRIVATE_KEY_FILE.

Run installation

Run Jupyter Notebook and follow the instructions in it to register DQ firewalls as Snowflake UDFs.

The notebook guides you through these steps:

Set up your local Python environment and connect to Ataccama ONE.
Fetch and download DQ firewalls from Ataccama ONE.
Connect to Snowflake.
Upload the firewalls to Snowflake and convert them to Snowflake UDFs.

For firewall selection options, see Firewall filtering.

Conflict strategies

Available from DQ Gates version 1.1.0.

Control how existing UDFs with the same name, signature, or firewall ID are handled during deployment.

Using built-in strategies. Pass a strategy name to register_firewall_in_snowflake:

skip_on_conflict
override_on_conflict
keep_both_or_skip

keep_both_or_override

Built-in strategy example

from ataccama_one_snowflake import register_firewall_in_snowflake

register_firewall_in_snowflake(
    session=session,
    firewall_zip_file_path="my_firewall.zip",
    stage_location="@stage",
    udf_name="validate_customers",
    conflict_strategy="keep_both_or_override",
)

Using custom strategies. Supply a callable that returns a ConflictResolution for fine-grained control.

The metadata used for conflict resolution is embedded in the Snowflake UDF comment. Avoid editing comment fields manually, as this breaks future conflict checks.

You can supply your own additional metadata when registering a firewall — it will be merged into the stored comment without overwriting the fields that conflict strategies rely on.

Custom strategy example

from ataccama_one_snowflake import ConflictResolution, register_firewall_in_snowflake

def drop_old_firewalls(conflicts, candidate_name, metadata) -> ConflictResolution:
    drops = []
    for udf in conflicts:
        created = udf.get("created")
        if created and created < metadata["threshold"]:
            drops.append(f"DROP FUNCTION IF EXISTS {udf['udf_name']}({udf['arg_sig']})")
        else:
            return ConflictResolution(
                abort=True,
                reasons=["Recent firewall already exists"],
            )
    return ConflictResolution(pre_sql=drops)

register_firewall_in_snowflake(
    session=session,
    firewall_zip_file_path="my_firewall.zip",
    stage_location="@stage",
    udf_name="validate_customers",
    conflict_strategy=drop_old_firewalls,
)

If your custom strategy needs additional information (for example, creation timestamps or owner details), query INFORMATION_SCHEMA.FUNCTIONS before calling register_firewall_in_snowflake and pass the enriched data into your resolver.

Verify the deployment

After the notebook finishes, verify that the UDFs were created in your target schema:

SHOW USER FUNCTIONS IN SCHEMA <your_database>.<your_schema>;

You should see functions named atc_dq_<firewall-name> for each deployed firewall.

To start evaluating data, continue with Evaluate Data Quality in Snowflake with UDFs.

Was this page useful?