Move DQ Rules and Monitors Between Environments

The REST API is currently in Early Access Preview. The API specification and endpoints might change before being marked as stable.

We recommend testing thoroughly and being prepared to adapt to potential changes in future releases.

This guide shows you how to move DQ rules and monitor configurations between environments, for example, promoting them from development through test to production. It covers both the first migration to an environment that has no DQ configuration yet, and the later propagation of individual changes, such as an added rule, a changed threshold, or a renamed rule.

Before using the Data Quality API, configure authentication as described in API Authentication. Your credentials must be valid for both the source and the target environment.

How migration works

There is no dedicated migration or sync endpoint.

You combine the existing endpoints in your own tooling or migration script, which determines what to move and applies it to the target environment.

The approach depends on whether the rule or monitor already exists on the target environment:

The rule or monitor does not exist yet: Create it on the target. See Migrate configuration for the first time.
The rule or monitor already exists: Compare both environments and update the target. See Propagate changes.
The rule or monitor was deleted on the source: Decide how to handle its counterpart on the target. See Propagate deletions.

A single migration run often involves more than one of these cases. In such a case, migrate all rules first (create missing rules, propagate changed ones), and only then create and update the monitors. Monitors reference rules, so every rule a monitor applies must already exist on the target environment.

Neither approach requires moving the entire DQ configuration: you can move any subset of rules and monitors you choose.

Before you start

Match rules and monitors between environments

Ataccama ONE assigns the URN when a rule or monitor is created, so the same rule or monitor has a different URN in each environment. Your tooling therefore needs to pair a rule or monitor on the source environment with its counterpart on the target: either through a mapping that you maintain (for example, the source-to-target URN pairs your tooling saves from the create responses, see Step 2: Create the rules on the target environment), or by matching a property such as the name.

Rule and monitor names are not unique, so name-based matching can silently pair the wrong rules or monitors. Prefer the mapping maintained by your tooling; if you match by name, verify each pairing before you apply changes.

Rule or monitor definitions contain additional URNs that also require pairing:

A monitor definition contains the URN of its catalog item and the URNs of the catalog item attributes used in its rule mappings. When the monitor applies cross-table rules, its definition also contains the URNs of the joined catalog items and their attributes (in the join configurations and connected mappings).
A rule definition can also contain URNs, for example in its conditions or stewardship assignment.

Check the URNs in each definition and replace any that point to the source environment with their target equivalent before you send it:

To find the target equivalents of catalog item and catalog item attribute URNs, list the catalog items and their attributes on the target environment; see Catalog Items API.
To find the target equivalents of DQ dimension and dimension result URNs (dqDimensionUrn, resultUrn), list the DQ dimensions on the target environment; each dimension includes its results. See List DQ dimensions. The UUIDs are not guaranteed to match across environments, so pair the dimensions by name, not by UUID.
To find the target equivalent of a DQ transformation URN (transformationUrn), list the DQ transformations on the target environment. See List DQ transformations.

Transformation names are not unique and the UUIDs are not guaranteed to match across environments, so pair the transformations by name and input data type.
The URN of a group (groupUrn) cannot be retrieved through the API yet, and its UUID is not guaranteed to match across environments. Build the URN yourself, always from the target environment: in Global settings > User management > Groups, select the group, copy the UUID from the URL (…/userManagement/groups/{uuid}/), and use it as the {uuid} part of the URN (for the URN format, see URN Reference).

To access Global settings, select the Ataccama logo.

Check the prerequisites

Publish what you want to move. Only published versions are relevant for migration: publish any drafts on the source environment first.
Make sure the catalog items exist on the target. The migrated configuration depends on catalog items in two ways: monitors are attached to a catalog item, and rules can use catalog items as reference data.

Before the migration, make sure these catalog items exist on the target environment (otherwise, the rules import successfully, but evaluations fail):
- For catalog items imported from data sources, connect the sources and import them.
- For catalog items managed through Reference Data, create and publish the tables in the web application: the public API cannot create them (see Create Reference Data Tables and Publish and Approve Reference Data).
Move rules before monitors. Monitors reference the rules they apply, so the rules must exist on the target environment first.
Avoid configuration changes during the migration. If rules or monitors are created or deleted on the source environment while the migration runs, the migration might miss some of them.

Rules built with Condition Builder

Condition Builder logic cannot be copied

In the web application, rule logic is defined either in Condition Builder or as an Advanced Expression. In the API, rule logic is always an advanced expression, stored in the stringExpression field of each condition.

Logic built in Condition Builder is stored separately, as a JSON string with internal IDs for the rule’s attributes, parameters, and variables. These IDs are generated when the rule is created and can’t be used in another environment.

When you retrieve a rule, the conditions built in Condition Builder have an empty stringExpression. No endpoint returns their logic, not even as an advanced expression. If you see a non-empty value instead, it is a leftover of an earlier API request that tried to set the expression. See Limitations of the DQ rules API.

Move rules built with Condition Builder

Moving these rules requires rewriting the logic manually as an advanced expression:

Find the rules with Condition Builder logic. Your tooling can search published rules for conditions with an exactly empty stringExpression. Compare without trimming (a whitespace-only expression is valid on an advanced-expression condition).

The one edge case that escapes the search is a Condition Builder condition whose stringExpression was set by an earlier API request and is therefore not empty. If that might have happened in your environment, verify the rule logic in the web application.
Check the rule logic in the web application on the source environment.
Recreate the logic as an equivalent expression in ONE expression language.
Use that expression as the stringExpression of the rule condition when you create the rule on the target environment.

A rule created this way is an advanced-expression rule. On the target environment, it is edited as an Advanced Expression in the web application, and whenever the rule logic changes on the source environment, propagating the change requires rewriting the expression manually again.

Term rule instances

Term rule instances cannot be moved through the Data Quality API (see Manage DQ Monitors via API). The underlying term assignments to the catalog item’s attributes do not transfer between environments automatically.

Term rule instances exist only on the catalog item’s primary DQ monitor, so secondary monitors are not affected (see Add DQ Rules to Terms).

To restore the monitor’s term rule instances on the target environment, recreate the assignments there:

On the source environment, retrieve the monitor.
Read each term’s termUrn and termName from the terms array in termRuleInstances.
On the target environment, find the URN of each term by listing the terms, and create the terms that are missing (see Create terms in bulk).
Link the DQ rules to the terms in the web application, before you assign the terms: rules linked after assignment are not applied retroactively (see Rules apply on first assignment). If a term is already assigned to the target attributes, add the rules directly to those attributes instead.
Assign the terms to the target catalog item attributes with POST /catalog/v1/attributes/term-assignments/batch/assign. Assignments created through the API have the source MANUAL. A nonexistent termUrn fails as a per-item error in the 207 response; the other assignments in the request still apply.
After the first assignment, the term rule instances appear on the primary monitor automatically. To verify, retrieve the monitor and check termRuleInstances.

Migrate configuration for the first time

Use this approach for rules and monitors that do not exist on the target environment yet.

All primary monitors already exist on the target environment: every catalog item has one. To move a primary monitor’s configuration, follow Propagate monitor changes with primary as the dqMonitorUrn.

Step 1: Retrieve the configuration from the source environment

List the rules and monitors on the source environment.

For monitors, use the catalogItemUrn parameter to restrict the result to a single catalog item.
Request: List rules on the source environment
```
curl -X GET "https://{source-environment}.ataccama.one/api/data-quality/v1/rules?size=100" \
  -H "Authorization: Bearer {source_access_token}"
```
Request: List monitors on the source environment
```
curl -X GET "https://{source-environment}.ataccama.one/api/data-quality/v1/dq-monitors?size=100" \
  -H "Authorization: Bearer {source_access_token}"
```
The list responses are paginated. When the source environment has more rules or monitors than one page returns, follow the meta.next cursor with the after parameter until you have retrieved them all.

See List rules and List monitors.
Retrieve each rule and monitor you want to move, using the URNs returned in the list responses:
Request: Retrieve a rule from the source environment
```
curl -X GET "https://{source-environment}.ataccama.one/api/data-quality/v1/rules/{ruleUrn}" \
  -H "Authorization: Bearer {source_access_token}"
```
To retrieve a monitor, you also need the URN of its catalog item. Both URNs are included in the monitor’s list entry.
Request: Retrieve a monitor from the source environment
```
curl -X GET "https://{source-environment}.ataccama.one/api/data-quality/v1/catalog-items/{catalogItemUrn}/dq-monitors/{dqMonitorUrn}" \
  -H "Authorization: Bearer {source_access_token}"
```
Both endpoints return the published version by default, which is the version you migrate. The retrieved definitions are the input for the next two steps.

For details, see Retrieve a rule and Retrieve a monitor.

To move only a subset of your configuration, filter the retrieved rules and monitors in your tooling before you continue.

Step 2: Create the rules on the target environment

Create the rules with the batch endpoint, which accepts up to 100 rules per request. To move more rules, split them into multiple requests. Batch-created rules are published directly, so no separate publish step is needed.

Prepare a JSON file that contains a data array with the rules you retrieved in Step 1, one object per rule. In the following example, that file is rules.json.

Each rule must be complete: batch-created rules must meet the publish requirements.

To pair each created rule with its source later, add an id field to each object in the data array and set it to the source rule’s URN. The response returns the id in the matching result.
In each rule, prefix every sub-entity key with new_, at every level: an input group key 27f91a25-… becomes new_27f91a25-…, and the same applies to the inputs inside the group.

The create endpoints reject raw UUID keys with a validation error.

In each rule, replace the source values with their target equivalents (see Match rules and monitors between environments):
- dqDimensionUrn in the implementation, and resultUrn in each condition and in the fallback: The DQ dimension and dimension results on the target environment.
- transformationUrn in the transformations of a variable: The DQ transformation on the target environment.
- groupUrn in the stewardship assignment: The group on the target environment.

Create the rules with the batch endpoint:

Request: Create rules on the target environment

curl -X POST "https://{target-environment}.ataccama.one/api/data-quality/v1/rules/batch/create" \
  -H "Authorization: Bearer {target_access_token}" \
  -H "Content-Type: application/json" \
  -d @rules.json

Save the URNs from the response: you need them to assign the rules to monitors in the next step.

The response reports the outcome of each item separately and returns the URN assigned on the target environment, in the same order as the request items.

Response: Target URNs for the created rules

{
  "meta": { "total": 2, "success": 2, "fail": 0 },
  "results": [
    {
      "id": "urn:ata:{source-tenant}:data-quality:rule:111e4567-...",
      "status": "OK",
      "data": {
        "urn": "urn:ata:{target-tenant}:data-quality:rule:888e4567-..."
      }
    },
    {
      "id": "urn:ata:{source-tenant}:data-quality:rule:222e4567-...",
      "status": "OK",
      "data": {
        "urn": "urn:ata:{target-tenant}:data-quality:rule:999e4567-..."
      }
    }
  ]
}

For details, see Create rules in batch.

Step 3: Create the monitors on the target environment

Create the monitors with the batch endpoint, which accepts up to 100 monitors per request. To move more monitors, split them into multiple requests. Batch-created monitors are published directly.

Prepare a JSON file that contains a data array with the monitors you retrieved in Step 1, one object per monitor. In the following example, that file is monitors.json.
In each monitor, prefix every sub-entity key with new_, in the same way as for the rules in Step 2.

The monitor sub-entities are the rule instances, mappings, connected mappings, thresholds, parameter values, export definitions, join configurations, and foreign-key pairs.
In each monitor, replace the source values with their target equivalents (see Match rules and monitors between environments):
- catalogItemUrn: The catalog item on the target environment.
- ruleUrn in each rule instance: The rule URN returned in Step 2.
- catalogItemAttributeUrn in each mapping: The catalog item attribute on the target environment.
- For monitors that apply cross-table rules:
  - connectedCatalogItemUrn in each join configuration: The joined catalog item on the target environment.
  - localAttributeUrn and connectedAttributeUrn in each foreign-key pair: The attributes of the monitor’s catalog item and of the joined catalog item on the target environment.
  - connectedCatalogItemAttributeUrn in each connected mapping: The attribute of the joined catalog item on the target environment.
  - joinConfigurationKey in each connected mapping: The key of the join configuration within the same monitor. The value does not point to the source environment: instead of replacing it, prefix it with new_. This keeps it pointing to its join configuration (whose key you already prefixed with new_).

Create the monitors with the batch endpoint:

Request: Create monitors on the target environment

curl -X POST "https://{target-environment}.ataccama.one/api/data-quality/v1/dq-monitors/batch/create" \
  -H "Authorization: Bearer {target_access_token}" \
  -H "Content-Type: application/json" \
  -d @monitors.json

For details, see Create monitors in batch.

Next, confirm that the migrated configuration works on the target environment: see Verify the result.

Propagate changes

Use this approach for rules and monitors that already exist on the target environment, for example when a rule was added to a monitor, a threshold was changed, or a rule was renamed on the source environment.

The pattern is the same for rules and monitors: retrieve the current state from both environments, compare them, patch the target, and publish. You define in your own tooling how to compare the source and target and what to apply.

Patching opens a new draft on the target: the change does not take effect in evaluations until you publish it. If the target already has an unpublished draft, the update applies on top of it, and publishing includes those earlier draft changes as well.

Unlike for creation, there is no batch endpoint for updates: patch each changed rule and monitor individually. This also makes the pattern inherently selective: apply it only to the rules and monitors you want to update and leave the rest untouched.

Propagate monitor changes

Retrieve the monitor configuration from the source environment:
Request: Retrieve the monitor from the source environment
```
curl -X GET "https://{source-environment}.ataccama.one/api/data-quality/v1/catalog-items/{catalogItemUrn}/dq-monitors/{dqMonitorUrn}" \
  -H "Authorization: Bearer {source_access_token}"
```
For the primary monitor, use primary as the dqMonitorUrn on both environments.
Retrieve the corresponding monitor from the target environment.
Compare the two configurations and identify the changes.
Apply the changes to the target monitor. Replace any source-environment URNs with their target equivalents (see Match rules and monitors between environments). See Update a monitor.

Send each changed sub-entity, such as a rule instance, under its ID on the target monitor, and include only the fields that changed. Sub-entities you do not mention stay unchanged. To point a rule instance at a different rule, delete it and create a new one: its ruleUrn cannot be changed.
Publish the target monitor. See Publish a monitor.

Monitor propagation script example

For each monitor pair in the mapping, the following script retrieves the published version of the monitor from both environments, compares the two, patches the target monitor when they differ, and publishes the change. The source-to-target pairing (monitor_mapping) and the comparison logic (build_monitor_patch) are the parts your own tooling provides. Because a monitor is addressed through its catalog item, each side of the mapping is a pair of catalog item and monitor URNs.

Script example: Propagate monitor changes to the target environment

import requests

# SOURCE_URL and TARGET_URL are the API base URLs including the /api path,
# for example https://{environment}.ataccama.one/api

def get_monitor(base_url, token, catalog_item_urn, monitor_urn):
    response = requests.get(
        f"{base_url}/data-quality/v1/catalog-items/{catalog_item_urn}/dq-monitors/{monitor_urn}",
        headers={"Authorization": f"Bearer {token}"},
    )
    response.raise_for_status()
    return response.json()

# Source-to-target pairing maintained by your tooling:
# each side is a (catalogItemUrn, dqMonitorUrn) pair
for source, target in monitor_mapping.items():
    source_monitor = get_monitor(SOURCE_URL, source_token, *source)
    target_monitor = get_monitor(TARGET_URL, target_token, *target)

    # Your comparison logic: select the changed fields and replace any
    # source-environment URNs (rule URNs, rule attribute URNs, catalog
    # item attribute URNs) with their target equivalents.
    # Send the changed fields under the sub-entity IDs of the TARGET monitor:
    # anything the patch does not mention stays unchanged.
    patch = build_monitor_patch(source_monitor, target_monitor)
    if not patch:
        continue

    target_item_urn, target_monitor_urn = target
    requests.patch(
        f"{TARGET_URL}/data-quality/v1/catalog-items/{target_item_urn}/dq-monitors/{target_monitor_urn}",
        headers={"Authorization": f"Bearer {target_token}"},
        json=patch,
    ).raise_for_status()

    requests.post(
        f"{TARGET_URL}/data-quality/v1/catalog-items/{target_item_urn}/dq-monitors/{target_monitor_urn}/publish",
        headers={"Authorization": f"Bearer {target_token}"},
    ).raise_for_status()

    print(f"Propagated: {source_monitor['name']}")

Propagate rule changes

Retrieve the rule from the source environment. See Retrieve a rule.
Retrieve the corresponding rule from the target environment.
Compare the two definitions and identify the changes.
Apply the changes to the target rule. Replace any source-environment URNs with their target equivalents (see Match rules and monitors between environments). See Update a rule.

Send changes to existing sub-entities, such as input groups and inputs, under their IDs on the target rule, never under the source IDs. Use a new_<id> key only for sub-entities that are genuinely new: it creates a sub-entity under a new URN, and recreating an existing input attribute this way breaks any DQ monitor mappings referencing it.
Publish the target rule. See Publish a rule.

Rule propagation script example

For each rule pair in the mapping, the following script retrieves the published version of the rule from both environments, compares the two, patches the target rule when they differ, and publishes the change. The source-to-target pairing (rule_mapping) and the comparison logic (build_patch) are the parts your own tooling provides.

Script example: Propagate rule changes to the target environment

import requests

# SOURCE_URL and TARGET_URL are the API base URLs including the /api path,
# for example https://{environment}.ataccama.one/api

def get_rule(base_url, token, rule_urn):
    response = requests.get(
        f"{base_url}/data-quality/v1/rules/{rule_urn}",
        headers={"Authorization": f"Bearer {token}"},
    )
    response.raise_for_status()
    return response.json()

# Source-to-target URN pairing maintained by your tooling
for source_urn, target_urn in rule_mapping.items():
    source_rule = get_rule(SOURCE_URL, source_token, source_urn)
    target_rule = get_rule(TARGET_URL, target_token, target_urn)

    # Your comparison logic: select the changed fields and replace
    # any source-environment URNs with their target equivalents
    patch = build_patch(source_rule, target_rule)
    if not patch:
        continue

    requests.patch(
        f"{TARGET_URL}/data-quality/v1/rules/{target_urn}",
        headers={"Authorization": f"Bearer {target_token}"},
        json=patch,
    ).raise_for_status()

    requests.post(
        f"{TARGET_URL}/data-quality/v1/rules/{target_urn}/publish",
        headers={"Authorization": f"Bearer {target_token}"},
    ).raise_for_status()

    print(f"Propagated: {source_rule['name']}")

Propagate deletions

A rule or monitor deleted on the source environment still exists on the target environment. Your tooling detects such cases through the mapping: the source side of a pair no longer exists.

How to handle the target counterpart is your decision:

Mirror the deletion: Delete the counterpart on the target environment. Deleting a rule also removes its rule instances from every monitor that applies it. See Delete a rule and Delete a monitor.
Keep the rule, remove the assignment: Update the monitors that apply the rule and delete its rule instances by setting them to null (see Propagate monitor changes). The rule stays on the target environment but is no longer evaluated.
Keep the rule, pause its evaluation: Update the monitors that apply the rule and set enabled: false on its rule instances (see Propagate monitor changes). The rule instances and their configuration stay on the monitors, but their DQ evaluation is paused; set enabled: true to resume it later.
Leave the counterpart unchanged: Keep the rule or monitor on the target, for example when the target environment intentionally keeps checks that were removed on the source. Record the decision in your tooling so that the pair is not flagged again on the next run.

Deletion through the API is immediate and permanent. There is no draft to review or publish.

Propagate changes back to the source environment

The retrieve-compare-patch-publish pattern also works in reverse, for example to roll back a change or to copy a fix made directly on production back to the source environment. Follow Propagate monitor changes or Propagate rule changes with the source and the target environment swapped.

The API does not track where a change originated: each environment only has the current state of its rules and monitors. When the same rule changed on both environments, your tooling must therefore decide which version wins, for example by always preferring one environment or by flagging the conflict for review.

Next, confirm that the updated configuration works on the target environment: see Verify the result.

Verify the result

Run a DQ evaluation on the target environment to confirm that the configuration works end to end.

Start the processing on a migrated or updated monitor:

Request: Start the processing on the target environment

curl -X POST "https://{target-environment}.ataccama.one/api/data-quality/v1/catalog-items/{catalogItemUrn}/dq-monitors/{dqMonitorUrn}/processings" \
  -H "Authorization: Bearer {target_access_token}"

Poll for completion and retrieve the results. See Run DQ Evaluation from Pipelines.
Check that:
- The processing finished successfully.
- Every migrated rule appears in ruleInstanceResults in the DQ results response (see Retrieve results for a single monitor).

Differences in pass and fail counts between environments usually reflect different data, not a failed migration. Check which rules were applied, not how many records passed.

Troubleshooting

404 Not Found when retrieving a rule or monitor on the target

The rule or monitor does not exist on the target environment. Create it instead of patching: see Migrate configuration for the first time.

VALIDATION_ERROR status in the batch response

The rule or monitor is incomplete, or a URN in the item belongs to the source environment. Check the item error in the batch response, and resolve all URNs to the target environment.

Migrated monitor has no rules applied

The rule instances reference rules or attributes that do not exist on the target environment. Check that:

The rules were created on the target first.
Each ruleUrn is the target rule URN returned when creating the rules (see Step 2: Create the rules on the target environment).
The mappings use target catalog item attribute URNs.

Fix the monitor with a PATCH request (see Update a monitor), or delete it and create it again.

Rule or monitor published without an error, but the change is missing

A change is applied in two steps: the PATCH request stores it as a draft, and publishing applies the draft. When the PATCH request fails, or goes to a different environment than the publish request, there is no draft with your change to apply. Publishing a rule or monitor without a draft reports no error and returns the currently published version.

Verify the PATCH response, and make sure both requests were sent to the same environment.

Evaluation uses the old version of an updated rule

The rule was left in draft. Publish the rule on the target environment.

Evaluation fails for a rule that uses a catalog item as reference data

The referenced catalog item is missing on the target environment or, for catalog items managed through Reference Data, not published; the rule itself imports successfully regardless.

Connect and import the data source with the catalog item, or, for catalog items managed through Reference Data, create and publish the table in the web application. Then start the processing again.

Rule built in Condition Builder does not work on the target

The rule logic contains internal IDs that can’t be used on the target environment. Rewrite the logic as an advanced expression before migration: see Rules built with Condition Builder.

API reference

For detailed API reference, including all endpoints, parameters, request and response schemas, and examples, see Data Quality API specification.

Next steps

Manage DQ Rules via API: Manage DQ rules in detail.
Manage DQ Monitors via API: Manage DQ monitor configuration in detail.
Run DQ Evaluation from Pipelines: Run DQ evaluation from your pipelines.

Was this page useful?