Split Step
Splits the data flow into three streams based on a ONE expression condition, while also providing a copy of all records.
|
This step is available for standalone plans, embedded plans, and transformation catalog items. It is not available for transformation rules. |
Overview
The Split step evaluates each record against a condition and routes it to separate outputs while also providing a complete copy of all records. This is useful when you need to process subsets of data differently while still maintaining access to the full dataset.
Use this step to:
-
Route records to different processing paths while keeping a master copy.
-
Create separate branches for valid and invalid records while logging all records.
-
Build complex workflows where some operations need all data and others need filtered subsets.
Configuration
Define the condition
-
In the step configuration, enter your condition using a ONE expression in the Condition field.
-
The expression must evaluate to a Boolean value (
trueorfalse). -
Connect downstream steps to the appropriate outputs.
Outputs
The Split step has three outputs:
out_true-
Records where the condition evaluates to
true. out_false-
Records where the condition evaluates to
false. out_all-
All input records, regardless of the condition result.
|
Records appear in both a conditional output ( |
Condition examples
email is not null and customer_id is not null
-
out_true: Valid records for normal processing. -
out_false: Invalid records for error handling. -
out_all: All records for audit logging.
region = 'EMEA'
-
out_true: EMEA records for regional processing. -
out_false: Non-EMEA records for different processing. -
out_all: All records for global reporting.
priority = 'High' or amount > 10000
-
out_true: High-priority items for expedited handling. -
out_false: Standard items for normal processing. -
out_all: All items for complete inventory tracking.
Example: Audit logging with conditional processing
A common pattern uses Split to maintain an audit trail while processing data differently based on conditions.
Process valid orders normally, send invalid orders for review, and log all orders for compliance.
-
Condition:
order_date is not null and customer_id is not null and total > 0 -
Connect
out_true→ Normal order processing → Database Output -
Connect
out_false→ Add error flags → Error table -
Connect
out_all→ Audit logging → Audit table
Split vs Condition vs Filter
| Aspect | Split | Condition | Filter |
|---|---|---|---|
Number of outputs |
3 ( |
2 ( |
1 (matching records only) |
All records output |
Yes (via |
No |
No |
Non-matching records |
Routed to |
Routed to |
Discarded |
Use case |
Route with full data access |
Route to different processing |
Remove unwanted data |
When to use Split vs Condition
Use Split when:
-
You need access to all records for logging, auditing, or reporting.
-
Different branches need the full dataset alongside filtered subsets.
-
You’re building compliance workflows that require complete data trails.
Use Condition when:
-
You only need to separate records into two groups.
-
You don’t need a copy of all records.
-
Simpler routing logic is sufficient.
Best practices
- Be mindful of data duplication
-
Records flow to both conditional outputs and
out_all. Design your plan to avoid unintended duplicate processing. - Use for audit scenarios
-
Split is ideal when you need to maintain a complete record of all data while also processing subsets differently.
- Connect all outputs
-
Even if you don’t need all three outputs, consider where each should route to avoid orphaned data paths.
- Document the purpose
-
Clearly describe why you’re using Split instead of Condition, especially regarding the use of
out_all.
Was this page useful?