Lead your team forward
OCT 24 / 9AM ET Register nowSnowflake Pushdown Processing
Query pushdown processing is performed directly in Snowflake, which greatly increases the performance when profiling your databases. In this scenario, Ataccama DPE does not process any data. Instead, only data samples and results are displayed ONE Web App.
The installation procedure has several steps:
-
Configure Snowflake to allow Ataccama to transfer functions to Snowflake
-
Configure ONE Web App to enable pushdown processing of databases
-
Configure DPE to allow for automatic synchronization of user-defined functions
Prepare pushdown processing in Snowflake
When Snowflake pushdown processing is used, Ataccama creates a stage to store functions and temporary tables during profiling. By default, Ataccama stores the stage, functions, and temporary tables in the database that is being processed but it is recommended to create a custom database. For production environments it is recommended to run the following SQL script. A Snowflake admin runs this script manually to allow Ataccama to process data:
-- create a role for pushdown processing
create role ataccama_pushdown;
-- grant this role to desired users or roles
-- this should cover all the connections within catalog data source items
grant role ataccama_pushdown to user john.doe;
grant role ataccama_pushdown to user jane.doe;
-- create working database
create database ataccama_pushdown_database;
grant imported privileges on database ataccama_pushdown_database to role ataccama_pushdown;
After the role is created, Ataccama ONE does the following:
-
Creates internal stage
_ATC_ONE_STAGE
and uploads jar files that enable additional functionality. -
Creates functions that reference the additional
jar
files. -
Shares the stage and functions to any additional roles defined in the script.
Configure pushdown in ONE
Follow these steps to enable Snowflake query pushdown:
-
Edit
fe/etc/application.properties
and add the following property to enable Pushdown Processing:ataccama.one.webserver.ff.props.profiling_snowflakePushdown=true
. -
Check Pushdown processing is enabled in the Snowflake connection in ONE.
-
(Optional) Specify Working database, where Ataccama searches for additional functions. By default, the database, which is being processed, is used. In the example above, we used the database
ataccama_pushdown_database
. -
(Optional) Specify the Snowflake roles that you want to share Ataccama objects with. In the example above, we used the role
ataccama_pushdown
.
DPE configuration
Configuration properties for pushdown profiling are specified in DPE Configuration.
Was this page useful?