ONE Desktop ONE DQG ONE MDM ONE RDM ONE Runtime Server
latest (15.1.0) 14.5.x 13.9.x 13.8.x 13.7.x 13.6.x 13.5.0 13.4.x 13.3.x 13.2.x
User Community Service Desk Downloads
If you can't find the product or version you're looking for, visit support.ataccama.com/downloads

Additional Configuration Options

In addition to the basic monitoring project configuration described in Monitoring Projects, you can also configure the following settings.

Parallel processing

By default, one DQ evaluation job is created for each data source.

Application admins can increase global parallelization, splitting each of these jobs into multiple smaller jobs to improve performance. This can be done both globally or per monitoring project.

If both are configured, project settings take priority over global settings.

Global configuration

Parallel processing is configured globally via the property plugin.monitoring-project.ataccama.one.monitoring-project.dq-processing-parallelism. See MMM Configuration for more information.

Project configuration

To configure parallel processing on a project-by-project basis:

  1. Open your project and using the three dots menu, select Advanced settings.

    Advanced settings

  2. In Parallel processing, select Split jobs from each source into.

  3. Define the required number of jobs.

    Subgroups
How does paralellism work?

Let’s imagine there are 20 catalog items in a project: five from source A and 15 from source B.

When DQ evaluation starts as a part of the project run, we first group catalog items by source, with one job being created for each group. So, in our example, by default, there are 2 groups, meaning two jobs.

  • Group A - 5 catalog items.

  • Group B - 15 catalog items.

If you changed the parallelism to three, each of these groups is further split into three. That means the final jobs look like this:

  • A1 - 2 catalog items.

  • A2 - 2 catalog items.

  • A3 - 1 catalog items.

  • B1 - 5 catalog items.

  • B2 - 5 catalog items.

  • B3 - 5 catalog items.

The higher the dq-processing-parallelism value is, the smaller the jobs are and the quicker the processing time.

Filter by attribute

It is possible to use attributes as filters on the Report tab when you are viewing results at the catalog item level. To do this:

  1. In the required project, select the Configuration & Results tab.

  2. Select a catalog item.

  3. For the required attribute either:

    • Select the Filter by option to enable the filter.

      Filter by

    • Use the dropdown in the Filter by column if you want to add a custom name for the filter:

      Filter by

      1. Select Use attribute as a filter for DQ results to enable filtering with the selected attribute.

      2. Specify a name for the filter. If you do not specify the filter name, the filter name is the same as the attribute.

  4. Publish the changes to the project.

Attribute filters are not visible on the screen from which you enable them. If filters are enabled, you can use them on the Report tab when looking at results for the relevant catalog items.
Report filter

Turn off rule suggestions

Power users can turn off rule suggestions in monitoring projects via the property plugin.monitoring-project.ataccama.one.rule-suggestions-enabled. For more information, see MMM Configuration.

Anomaly detection configuration

When running anomaly detection during monitoring projects, you are able to select whether you want to run the time-dependent model or time-independent model. You can also configure the desired sensitivity of selection. See Anomaly Detection in Monitoring Projects.

Update rule references

If your monitoring project contains mappings to rules that have since been edited, you are notified on the Configuration & Results screen.

If you have view-only permissions for monitoring projects, this notification is not visible.

  1. To update the rule instances to the latest version, select Update all.

    Update

    • If the update is successful, publish the changes to continue.

    • If there are validation errors you are notified and it is necessary to resolve the issues before you can continue working with the monitoring project.

      Validation errors can occur, for example, if an input attribute data type is changed, and if an input attribute was added or removed.

      Validation error

Invalid results samples

The invalid results sample provides an opportunity to see a sample of records that are non-compliant with the DQ checks applied.

To change the configuration of invalid results samples:

  1. Navigate to the Configuration & Results tab.

  2. Using the three dots menu in the Data Quality section, select Configuration of invalid results samples.

    Configuration of invalid samples

    1. In Configuration of Invalid Results Samples, do the following:

      1. Select Prepare invalid samples.

      2. Set the size of the sample per catalog item by entering the required number.

        Settings invalid samples

    2. Select Save to apply the changes.

If invalid results samples are disabled in a monitoring project, they are not generated at all (that is, not saved).
Where are invalid samples saved?

If invalid samples are enabled, they are stored in the drillthrough bucket in ONE Object Storage.

You can define how long the samples should be stored for both on a project basis or on a global basis. Samples are deleted from the storage accordingly. For more information, see MMM Configuration.

Was this page useful?