Ataccama 13.9.0 Release Notes

Time-dependent anomaly detection within monitoring projects has been improved both in terms of capability and user interaction. The model can now detect irregular or missing timestamps and subsequently exclude them from the calculation or impute suitable values. This makes anomaly detection on time series more accurate. In addition, if no periodicity is specified, the system can derive it from the timestamps.

You now also need to specifically select to run time-dependent anomaly detection, whereas previously this was just assumed when periodicity was provided. Furthermore, you can either provide a periodicity value yourself or leave it to the system to derive it.

In addition to the existing notifications available on monitoring projects, from 13.9.0 it is possible to configure notifications for specific catalog items, groups, or rule instances, based on data quality thresholds you define. As well as the increased granularity, further improvements include being able to notify individual users (not just groups and roles), and being able to use filters within the notification configuration.

Both the existing general notifications (monitoring failed, critical error in structure, new results, DQ issue, or anomalies) and the custom notifications, will be listed in the monitoring projects Notifications tab. From there, you can edit, turn them off or on, or remove them as required.

DQ rules can now be included in export plans and then imported into a new instance. For more information, see Import and Export.

In version 13.9 of ONE, we have extended the permissions functionality. Now you can configure the number of users with the create permissions, which allows non-admin users to create new assets in ONE. Create permissions let users create a new asset on the node page using the Create button. For users without the necessary permissions, the button is disabled.

This way, you can separate assets for various groups and ensure that all users have the appropriate access rights. Consider the following when working with new create permissions:

Since 13.9 the new full-text search is enabled on the whole application and is a good starting point for searching data entities and locating the area for the future detailed search if necessary. For more information, see Search. To learn how to configure Global Search, see full-text-search-configuration.adoc.

Amazon S3 bucket can now be accessed using AWS IAM roles.

With the new MANTA lineage schema more data sources can be configured for lineage. MANTA-integrated lineage is now available not only for catalog items from Microsoft SQL and PostgreSQL but also for Oracle and Teradata sources.

In versions 13.8 and earlier, Term Suggestions and the Metadata Management Module (MMM) communicated using a term suggestions database, which needed to be configured. Since version 13.9, the connection is enabled via the new Api service and it is no longer necessary to configure the database connection. If you are upgrading from an earlier version, the Api service needs to be enabled for 13.9. For more information, see ONE Gen2 Upgrade Guide.

As the Term Suggestions services have improved in stability, it should not be necessary to initiate Term Suggestions synchronization when working with the application as usual. The only exception to this is when modifying the metadata model, for example, by restoring database from a snapshot or by importing any initial content (such as default or demo).

After restoring the database from a snapshot, Term Suggestions need to be synchronized on the Global Settings > Term Suggestions tab:

The API changes resulting from the changes to DQ project and DQ catalog results storage meant that the ONE Metadata Reader step could no longer be used to access results. This release brings new steps that can be used in addition to the Metadata Reader step (or other inputs) to access DQ results:

Starting from 13.9.0, we have changed how getParameterValue function behaves. To retrieve the value of an environment variable, this option must now be explicitly configured for that variable.

To learn more about how to enable this when running a plan or debugging a step, see Commonly Used Functions, section Other operations.

The date picker in both MDM and RDM now allows easier selection of a particular date.

MDM now natively supports the ability to load additional libraries from a dedicated location in both on-premise and PaaS deployments. External driver libraries for the most common data providers and consumers have been tested. See Loading Additional Drivers for details.

Generating training pairs is now as fast as it was originally before the AI Matching persistence was introduced in version 13.8. In addition, a Remove Model button has been added to the more options menu that allows you to remove an existing AI Matching model and start a new one from scratch.

AI Matching now also supports standard Prometheus metrics for monitoring (see Monitoring Metrics).

MDM Server HTTP requests can be selectively logged using logging filters and server responses can be configured to contain security headers (see MDM Server Application Properties).

To determine your upgrade path for version 13.9.0, please see the MDM Upgrade Procedure and MDM 13.9.0 Upgrade Notes.

The date picker in both MDM and RDM now allows easier selection of a particular date.

There are no version specific upgrade notes for 13.9.0, to determine your upgrade path see RDM Upgrade Procedure.