Ataccama ONE Agentic ONE Desktop ONE DQ&C ONE MDM ONE RDM ONE Runtime Server
Downloads

Transformation Rules

A transformation rule is a reusable piece of data standardization logic, such as normalizing date formats, trimming whitespace, or standardizing casing.

You create rules in the rule library, then apply them to source catalog items via a preparation set to produce a standardized output. Anyone with access to the library can apply published rules to their data, so the same standardization can be reused consistently across your organization.

What are transformation rules

A transformation rule consists of two key components:

  • Metadata: Description, tags, and other information that helps users understand what the rule does and when to use it.

  • Logic: The underlying transformation logic that processes the data.

Transformation rules are similar to embedded plans but have key differences:

  • They include additional metadata that makes them directly applicable to transformation catalog items (TCIs).

  • They have exactly one Input and Output step.

  • Their steps never change the number of records (maintaining the same record count throughout the transformation).

They represent a level of abstraction that focuses on the business purpose rather than technical implementation.

Why use transformation rules

Transformation rules are designed to:

  • Enable business users to prepare and transform datasets without needing to define transformation logic directly.

  • Increase efficiency when applying common data transformations across multiple datasets.

  • Enforce data standardization across your organization.

  • Allow separation of concerns: Technical users can create rules while business users can apply them.

When to use transformation rules

Transformation rules are ideal for:

  • Data standardization: Apply consistent formatting to data like phone numbers, addresses, or dates.

  • Data cleansing: Remove invalid characters, fix common errors, or standardize values.

  • Data preparation: Create derived fields or labels from existing data.

  • Any repeatable transformation: Define once, apply many times.

Create a transformation rule

Create transformation rules manually or with AI assistance.

Manual creation

To create a transformation rule manually:

  1. Go to Data quality > Transformation rules.

  2. Select Create transformation.

  3. Select Manual Creation as the creation mode.

  4. Enter a descriptive Name for your rule.

  5. (Optional) Associate your rule with a terms from the business glossary.

  6. (Optional) Enter a clear Description that explains what the rule does and when to use it.

  7. Select Create.

Your new transformation rule is created in Draft state with pre-generated input and output steps.

AI-assisted creation

If Gen AI features are enabled, you can let AI generate a transformation rule based on a plain text description:

  1. Go to Data quality > Transformation rules.

  2. Select Create transformation.

  3. Select AI Assisted Creation as the creation mode.

  4. In the AI Prompt field, describe the transformation you want to create. For example: "Standardize phone numbers to E.164 international format. Parse country codes, remove special characters, and validate the number length."

  5. Select Create.

AI generates the complete rule logic, including input and output steps and the necessary transformation steps, based on your description. You can use this as a starting point and further customize the logic or metadata as needed.

Configure rule logic

Once you’ve created a transformation rule, you need to define its transformation logic:

In the transformation editor canvas:

  1. Note that input and output steps are already added to the canvas.

  2. Define inputs by clicking on the input step and specifying which attributes will be processed.

  3. Add transformation steps between the input and output steps.

  4. Connect steps by dragging from output ports to input ports.

  5. Configure each step by clicking on it and setting its properties.

  6. Configure the output step if needed for specific attributes.

  7. Validate your rule by selecting Validate plan.

  8. Test your rule with sample data using the data preview.

Transformation rules must not change the number of records. They should transform the input data without aggregating or filtering out records.

Supported transformation steps

Transformation rules can use the following step types:

  • Add attributes: Create new attributes derived from input attributes.

  • Transform data: Modify attribute values with expressions.

  • Delete attributes: Remove attributes that are not needed.

  • Condition: Split the data flow into two streams based on a condition (requires merging streams before output to maintain record count).

For detailed information about configuring each step type, see Data Transformation Steps Reference.

Best practices for transformation rules

Follow these guidelines to keep rules effective and easy to maintain:

  • Be specific: Create rules that do one thing well rather than trying to combine multiple transformations.

  • Test thoroughly: Plan for diverse test scenarios to handle edge cases.

  • Document clearly: Write descriptions that explain both what the rule does and when to use it.

  • Use meaningful names: Name rules to clearly indicate their purpose.

  • Tag effectively: Add relevant tags to make rules discoverable.

  • Organize logically: Create a consistent taxonomy for your transformation rules.

Consider creating a set of sample test data that includes edge cases for your rule. This helps ensure your rule handles unusual or unexpected data correctly before it’s used in production.

Add meaningful metadata

Good metadata is essential for making your transformation rules discoverable and usable:

  1. Return to the Overview tab.

  2. Enhance the rule Description with detailed information about:

    • What the rule does.

    • When to use it.

    • Example inputs and outputs.

  3. Add Terms if the rule relates to specific concepts in your Glossary.

Publish your transformation rule

When you’re satisfied with your rule:

  1. Select Publish from the upper-right menu.

  2. Your rule is now available for use across your organization.

Sufficient access level may be needed depending on user management settings of your organization.

Manage transformation rules

The rule library at Data quality > Transformation rules lists all transformation rules in your organization. From here you can:

  • Search rules by name.

  • Open a rule to review or edit its logic, metadata, or terms.

  • Republish a rule after making changes. Preparation sets using the rule pick up the new logic the next time their output is generated — no need to update each preparation set individually.

  • Validate listed rules to check for configuration issues.

A single rule can be applied across multiple source catalog items — define the logic once, use it everywhere.

Apply transformation rules

To use a published transformation rule, select it from the library while authoring a preparation set on a source catalog item.

To apply a rule:

  1. Go to the source catalog item you want to standardize.

  2. Open the Prepare data tab.

  3. Create a new preparation set or open an existing one.

  4. Inside the preparation set, select Add transformation rule.

  5. Choose your rule from the library.

  6. Configure any rule parameters and preview the output.

  7. Select Apply.

  8. Publish the preparation set.

  9. Select Create output to produce a new transformation catalog item with the rule applied.

For details on previews, multiple rules in one preparation set, modifying preparation sets, and creating output, see Prepare Data with Transformation Rules.

Example: Standardize bank transaction data

This example shows how to author two transformation rules to clean a bank_transactions table with missing country codes and inconsistent date formats.

Source data

transaction_id iban country transaction_date amount

TXN-001

GB29NWBK60161331926819

null

2024-03-15

1250.00

TXN-002

DE89370400440532013000

de

15.03.2024

890.50

TXN-003

FR7630006000011234567890189

null

03/15/2024

3400.00

TXN-004

NL91ABNA0417164300

NL

2024-03-16

210.75

TXN-005

ES9121000418450200051332

ES

16.03.2024

567.00

Issues:

  • country is null for some rows, lowercase for others, and uppercase for others.

  • transaction_date appears in three different formats: yyyy-MM-dd, dd.MM.yyyy, and MM/dd/yyyy.

  • The country code is always available in the first two characters of iban, so it can be extracted when missing.

Expected output:

transaction_id iban country transaction_date amount

TXN-001

GB29NWBK60161331926819

GB

2024-03-15

1250.00

TXN-002

DE89370400440532013000

DE

2024-03-15

890.50

TXN-003

FR7630006000011234567890189

FR

2024-03-15

3400.00

TXN-004

NL91ABNA0417164300

NL

2024-03-16

210.75

TXN-005

ES9121000418450200051332

ES

2024-03-16

567.00

Standardize transaction date

Normalizes transaction_date from three string formats (yyyy-MM-dd, dd.MM.yyyy, MM/dd/yyyy) to a consistent date type.

Step pipeline:

In any expression field, use AI assistance — describe what you want in plain language, for example: "Convert transaction_date string to a date type, handling formats yyyy-MM-dd, dd.MM.yyyy, and MM/dd/yyyy".

Extract country from IBAN

Derives the country code from the first two characters of iban when country is missing, and normalizes existing values to uppercase.

Step pipeline:

Apply the rules

After authoring and publishing both rules, apply them to the source bank_transactions catalog item via a preparation set:

  1. Open the Prepare data tab on the bank_transactions catalog item.

  2. Select Add preparation set.

  3. Name the preparation set Bank transactions clean and select Create.

  4. Inside the preparation set, select Add transformation rule.

  5. Choose Standardize transaction date and select Apply.

  6. Select Add transformation rule again.

  7. Choose Extract country from IBAN and select Apply.

  8. Preview the combined result with Preview final output, or check each rule individually with Preview by rule.

  9. Publish the preparation set.

  10. Select Create output and complete the dialog to produce a new TCI with the cleaned data.

The original bank_transactions catalog item is unchanged. The new TCI applies the rules on every access — no scheduling, no manual reruns.

For full details on the preparation set workflow, see Prepare Data with Transformation Rules.

See also

Was this page useful?