If you can't find the product or version you're looking for, visit support.ataccama.com/downloads

Ataccama 13.3.0 Technical Release Notes

This page contains the technical release notes detailing stability, performance, and observability improvements for the release of version 13.3.0 of the entire Ataccama ONE Gen2 platform. More information about each topic can be found by expanding the relevant section. In addition, in the section New and Newsworthy you can find a summary of key functionalities that have been introduced in this version.

For more detailed product updates, see Ataccama 13.3.0 Release Notes.

New and Newsworthy

The following section provides an overview of non-functional changes, such as changes in configuration and user experience.

AI Core:
- Improved processing frequencies in Anomaly Detection, with added row and distinct counts.
- Fixed number imprecisions in expected bounds in time series analysis.
- Added a job type for fetching metadata from MMM.
Data Processing Module (DPM):
- DQ filters are now correctly applied when displaying results.
- Improved logging for disconnected engines when DEBUG logging level is enabled.
- Improved the accuracy of sample profiling for MSSQL data sources.
- Lineage operations are now routed only to ** DPEs with enabled MANTA integration. DPEs are no longer falsely overloaded by metadata import jobs that cannot be restored.
- Added an optional configuration for MSSQL connections that does not use any locks when executing the queries configured.
- Fixed issues with canceling metadata import jobs.

New Features

Debugging Component Rules

Component rules can be evaluated from the platform UI, making it easier to decide whether the rule can be applied in a particular use case or not.

Reason for Invalidity in Components Rules

In addition to default VALID/INVALID outputs, it is possible to define the reason for invalidity for component rules in the web application. The same information is then also displayed in ONE Desktop.

Filtering DQ Results

You can filter DQ results in order to discover the root cause of detected issues. The project editor can select attributes that will be used as filters, which in turn allows users to filter DQ results based on specific attribute values.

Scheduling for Catalog Items

Scheduling can now be configured for all actions available on catalog items, such as full or sample profiling or importing. If the entity is partitioned, users can schedule actions for a specific partition only. The upcoming run is displayed on the catalog item. The overview of all scheduled events can be found in the Processing Center under Scheduled Jobs.

DQ Flow Configuration Available in UI

It is now possible to configure whether DQ is automatically performed in the scope of full or custom profiling directly from the ONE webapp interface. Profiling configurations can be updated by power users with application admin permissions.

Assigning Environments to Catalog Items

You can assign a specific label (purpose) to a catalog item or data instance, which allows you to use different data sets depending on the usage (for example, for testing and production purposes) and easily deploy the necessary data based on the environment or another criterium. Labels are configurable as needed – you can use different environment names (prod, dev, qa) or countries (UK, US, CZ).

Transferring Monitoring Project Configuration

The configuration of a monitoring project can be copied to another monitoring project if the structure of the mapped catalog items and rules is the same. This lets you test the project configuration on different data sets before the project reaches production without affecting business users. Projects can have different permissions, notifications, and schedules as these elements are not transferred.

Anomaly Inspector Improvements

We have improved the Anomaly Inspector functionality by adding frequency analysis visualization, attribute overview, and split filters, which allow users to filter results depending on whether an item is anomalous or not, as well as to display only anomalous charts of anomalous items.

UX Improvements

The 13.3.0 release comes with the following UX updates:

Reverting Entities to a Previous Version
- You can now easily revert an entity to one of its previous versions, which creates a new version based on the selected one.
Cancelling Jobs
- Jobs can be cancelled now from the platform UI. The feature is not currently available for MANTA jobs.
DPM Admin Console Improvements
- In DPM Admin Console, the Last Heartbeat of an engine is now correctly updated.
- Fixed duplicate labels for job details in DPM Admin Console that occasionally caused incorrect rendering of job details.

Stability Improvements

We have made the following changes to improve the platform stability:

AI Core:
- Updated a number of integrated third-party dependencies to newer versions. No user action is needed.
Data Processing Module (DPM):
- Event subscribers are now handled in separate threads to improve throughput and isolate their context. This also ensures that the thread context is not accidentally cleared as that can lead to authentication issues.

Performance Improvements

The platform performance has been improved in the following ways:

AI Core:
- Removed unnecessary package imports, which reduces memory consumption.
Data Processing Engine (DPE):
- We have enhanced platform performance during metadata import for JDBC data sources such as MySQL, MariaDB, Amazon Redshift, Teradata.
Metadata Management Module (MMM):
- Thanks to the improvements made to computed content queries, catalog items and attributes tabs are loaded faster. This also includes calculating the view for Run DQ button on glossary terms and showing catalog item relationships.
- Improved performance of Anomaly Inspector data fetching.

Observability Improvements

The following changes have been made in order to provide more insight into the platform’s functioning as well as help users optimize system resources:

On the platform-level, we have improved observability dashboards and made data sizes available in Prometheus. The Grafana health check dashboard (Ataccama > Health checks) provides the following information:

The health status of certain components. Currently, the dashboard covers MMM (backend), DPM, DPE.
The connection status between a component and its dependencies, for example, between MMM and DPM.
Infrastructure issues.

Historical information about component availability, allowing you to see if any component has been unavailable for some time.

In order to avoid potential impact on performance, health check metrics are not enabled by default. For more information, see configuration:reference:monitoring-configuration.adoc, sections DPM and DPE Health Check Metrics and MMM Metrics.

AI Core:
- Added logging information about the Configuration Service gRPC connection failures and reconnections.
- Better handling of invalid response codes and processing errors when submitting GraphQL requests.
- Added logging of authentication failure reasons for GraphQL requests.
- Removed wsgi_application logging for successful requests to the monitoring endpoint.
- Metrics names have been unified.
Data Processing Engine (DPE):
- Enabled health checks for the module as well as configuring their metrics.
Data Processing Module (DPM):
- Added metrics for the engine registry statistics.
- Added metrics for job submissions, thread pools, queue size of incoming requests, pre-processing duration, queue size of all jobs, which is now parametrized by job status, duration of uploading and downloading access files, delay between job status changes, gRPC message size.
- Enabled health checks for the module as well as configuring their metrics.

Was this page useful?