If you can't find the product or version you're looking for, visit support.ataccama.com/downloads

Import Monitoring Project Configuration

Test new rules on a catalog item in a monitoring project without risking unnecessary changes in a production project. Using monitoring project import, you can import only selected catalog items and also replace selected catalog items with alternatives.

Furthermore, on the attribute level, you can remap attributes (provided the new mapping fulfills data type constraints) or ignore selected attributes in the import.

Alternatively, copy entire monitoring project configurations. This is useful when, for example, you want to import configuration from testing to production project (to see this in action, see Monitoring project lifecycles).

Configuration import includes:

All checks (structure checks, anomaly detection, DQ checks)
Invalid samples
Reports structure
DQ filters

To import a monitoring project:

Select the project you want to import the configuration into. Using the three dots menu, select Import configuration.

On the Import monitoring project configuration screen, from the list provided, select the monitoring project from which the configuration should be used.

By default, items in the target project are overwritten during import. If a catalog item exists at target but not at the source, it is deleted.

Select Keep existing configurations of Catalog items that are only at target if you don’t want these items to be deleted.

Map catalog items in the source and target monitoring projects. Without further action, items in the source project overwrite those in the target project.

If required, adjust the mappings of the catalog items in the next step.
Edit the mappings as necessary. You can:
1. Manually map alternative catalog items. To do this, select Select different and choose from the list provided.
  
  You are notified if the structure of the selected catalog item does not match that of the source catalog item. If this happens, you can manually map alternative attributes (see b) .
2. Manually map alternative attributes. To do this, select Map attributes and use the dropdown to select available attributes.
  
  Alternatively, exclude selected attributes (see c).
3. Exclude individual attributes. To do this, clear the selected attributes as needed.
  
  If the removed attribute is used as an input attribute in multi-input rules, the associated rule is removed from the project.
  
  Removing attributes here affects post-processing plans and the report structure. Update the plans manually after importing.
Select Start import at the bottom of the screen. The configuration and the report settings are imported to the target monitoring project:

Monitoring project lifecycles

You can make use a combination of Purpose tags and of the import monitoring project configuration feature to simulate a multi-environment process (Configure > Test > Send for approval > Make public, and repeat) within one physical environment.

In summary:

Create a project, use tables from the non-production data source, assign rules, and test.
Create a second project and use the new Import configuration functionality, and import the configuration from the first project.
Map tables to their non-production instances:
- If catalog items have not been assigned a purpose tag, and this is the first import of the configuration, you need to map items manually.
- If the configuration has been previously imported, mapping is automatic (this can be overridden by the user).
- If catalog items have been assigned a purpose corresponding to that of the project, these items are selected.

Two physical environments can be created, but the intended use for the non-production environment should be only:

Testing new sources.
Testing new integration plans (for example, you have a workflow that reads or writes to API).
Testing model changes.
Testing service re-configurations (for example, you can change the settings in configuration files and check whether the platform remains stable).

These instances are not intended for:

Developing data quality monitoring projects and then migrating between them (that is, developing monitoring on DEV and then physically migrating to PROD). This would entail either manually propagating changes to the PROD environment, or creating migration scripts using built-in APIs, or custom APIs (usually implemented by plans and API steps).

The preferred method on multi-environments is one production environment for all metadata, and one non-production environment for model changes, API work, and testing. This differs to other products, for example, RDM and MDM, for which one production and two non-production environments are recommended.

Copying the configuration of a project means you can create, for example, a DEV and PROD instance of the same project, to test projects on non-production data and later exchange for production data.

This allows you to test rules and configurations, particularly how changes in rules affect monitoring projects, without affecting the production dashboard. It also means it is not necessary to configure the same project multiple times, which is highly valuable given that organizations can have large numbers of catalog items in a project with multiples rules for each table.

Create rules and connect them to your production and non-production data sets. Link the matching catalog items from each data set, and create two separate projects, for example, one for DEV and one for PROD. You can distinguish between these using the Purpose label.

It is only be necessary to configure one project, and you can then easily copy the configuration to the second project. In the guide that follows, we use the DEV and PROD example to illustrate this functionality.

Tutorial

Add both a DEV and a PROD purpose in List of Values. For more details, see Lists of Values.
1. In Global Settings > List of Values, select Add Data instance.
2. Provide a name (that is, DEV or PROD).
3. Select Save
Select an existing project to be used as the DEV instance, or configure one, for example, My project - dev.
In General information, add a purpose for the monitoring project: DEV.
Create a monitoring project for the production environment, for example, My project - prod. You do not need to configure this project.
In General information, add a purpose for the monitoring project: PROD.
Copy the monitoring project configuration for My project - dev to My project - prod, that is, from the development environment to the production environment according to the instructions found earlier in this article.

Was this page useful?