User Community Service Desk Downloads

Snowflake Pushdown Processing

Overview

Pushdown processing involves delegating data processing tasks (namely profiling and DQ evaluation) to your connected Snowflake warehouse instead of running them on Ataccama DPEs.

In this case, instead of transferring data into ONE for processing, ONE generates SQL queries that run directly within Snowflake, leveraging its computational resources. Only results and invalid samples, if configured, are returned to ONE.

Snowflake pushdown processing is supported for profiling and DQ evaluation jobs.

Pushdown profiling

Profiling runs on pushdown for all Snowflake connections. There are no limitations when running profiling in Snowflake pushdown compared to profiling in ONE.

Pushdown for DQ evaluation

Pushdown processing for DQ evaluation can be enabled or disabled as required, as some limitations exist. See When to Use Snowflake Pushdown for DQ Evaluation.

Transformation catalog items and SQL catalog items cannot be processed in pushdown.

Why use pushdown processing?

Using pushdown processing can lead to improvements in performance, security and infrastructure, and subsequently, scalability.

  • Performance. Evaluate millions of records in seconds.

  • Security. Data processing is executed directly in Snowflake, and results (and invalid samples, if configured) are returned. This means there is no transfer of data required or any external services or servers; all processes run on secured and governed Snowflake warehouses.

  • Infrastructure. Utilize your existing infrastructure to evaluate data—no need to set up and manage new processing servers.

How to proceed

Was this page useful?