If you can't find the product or version you're looking for, visit support.ataccama.com/downloads

Snowflake Pushdown Processing

This is an experimental feature.

Query pushdown processing is performed directly in Snowflake, which greatly increases the performance when profiling your databases. In this scenario, Ataccama DPE does not process any data. Instead, only data samples and results are displayed ONE Web App.

The installation procedure has several steps:

Configure Snowflake to allow Ataccama to transfer functions to Snowflake
Configure ONE Web App to enable pushdown processing of databases
Configure DPE to allow for automatic synchronization of user-defined functions

Prepare pushdown processing in Snowflake

When Snowflake pushdown processing is used, Ataccama creates a stage to store functions and temporary tables during profiling. By default, Ataccama stores the stage, functions, and temporary tables in the database that is being processed but it is recommended to create a custom database. For production environments it is recommended to run the following SQL script. A Snowflake admin runs this script manually to allow Ataccama to process data:

-- create a role for pushdown processing
create role ataccama_pushdown;

-- grant this role to desired users or roles
-- this should cover all the connections within catalog data source items
grant role ataccama_pushdown to user john.doe;
grant role ataccama_pushdown to user jane.doe;

-- create working database
create database ataccama_pushdown_database;
grant imported privileges on database ataccama_pushdown_database to role ataccama_pushdown;

After the role is created, Ataccama ONE does the following:

Creates internal stage _ATC_ONE_STAGE and uploads jar files that enable additional functionality.
Creates functions that reference the additional jar files.
Shares the stage and functions to any additional roles defined in the script.

Configure pushdown in ONE

Follow these steps to enable Snowflake query pushdown:

Edit fe/etc/application.properties and add the following property to enable Pushdown Processing: ataccama.one.webserver.ff.props.profiling_snowflakePushdown=true.
Check Pushdown processing is enabled in the Snowflake connection in ONE.
(Optional) Specify Working database, where Ataccama searches for additional functions. By default, the database, which is being processed, is used. In the example above, we used the database ataccama_pushdown_database.
(Optional) Specify the Snowflake roles that you want to share Ataccama objects with. In the example above, we used the role ataccama_pushdown.

DPE configuration

Configuration properties for pushdown profiling are specified in DPE Configuration.

Was this page useful?