ONE Desktop ONE DQ&C ONE MDM ONE RDM ONE Runtime Server
latest (15.4.0) 15.3.0 15.2.0 15.1.0 14.5.x 13.9.x 13.8.x 13.7.x 13.6.x 13.5.0 13.4.x 13.3.x 13.2.x
User Community Service Desk Downloads
If you can't find the product or version you're looking for, visit support.ataccama.com/downloads

Ataccama 13.2.0 Technical Release Notes

This page contains the technical release notes detailing stability, performance, and observability improvements for the release of version 13.2.0 of the entire Ataccama ONE Gen2 platform. More information about each topic can be found by expanding the relevant section. In addition, in the section [New and newsworthy] you can find a summary of key functionalities that have been introduced in this version.

For more detailed product updates, see Ataccama 13.2.0 Release Notes.

New and Newsworthy

The following section provides an overview of non-functional changes, such as changes in configuration and user experience.

  • AI Core:

    • We now provide a means of transparently upgrading the AI Core module by restarting a new microservice called Upgrade. The microservice first needs to be enabled in the configuration.

    • You can now set the maximum number of attributes for the Neighbors microservice. The microservice refuses to start if MMM contains more attributes than the specified value.

    • There is a new microservice/K8s job for downloading metadata from MMM to ONE Object Storage (MinIO or S3) called Metadata Fetcher (currently a prototype).

    • We have redesigned the Anomaly Detection API to accept more generic inputs.

    • Fixed support for CentOS 7 (compilation of 3rd-party libraries).

    • Added a script (Bash) for checking the health status of all AI Core microservices at once with a human-readable output (useful for validation of initial deployment and troubleshooting).

  • Data Processing Module (DPM):

    • Added a set of properties for configuring the engine registry:

      • ataccama.one.dpm.registry.channel-provider-ttl

      • ataccama.one.dpm.registry.inactivate-scheduler-period

      • ataccama.one.dpm.registry.automatically-inactivate-engines-not-seen-for

    • Increased the values of default timeout for jobs (plugin.executor.ataccama.one.submit-timeout, plugin.executor.ataccama.one.job-at-dpm-timeout).

  • Metadata Management Module (MMM):

    • Added a configurable gRPC server executor thread pool (properties ataccama.server.grpc.executor.*).

    • MMM now exposes the Spring Actuator on two separate ports (by default: 8021 and 8024). In the current version, it is necessary to scrape the Prometheus endpoint exposed on both ports.

Platform Configuration

New Application Upgrade Center

The upgrade process is now more streamlined thanks to the Upgrade Center. The Upgrade Center can be accessed through the ONE webapp and it comes with options to easily back up and restore user data and metadata model. In addition, users can validate the state of the application after upgrading to a new platform version. For more information, see System Changes.

Permissions and Workflow Configuration Now Available in UI

It is now possible to configure permissions, user roles, and metadata workflows directly from the ONE webapp interface. While this makes the configuration more accessible, making any of the changes mentioned can only be done by power users.

Managing Default Top-Level Access for Entities

You can now modify the top-level configuration of any array-embedded entity, such as sources, rules, terms, in order to define default access permissions, which are then propagated to all root entities (that is, all sources, rules, or terms). When the default permissions are set on the top-level entities, all changes are similarly propagated to all the available entities in the list.

Improved Navigation

The left-navigation menu is now easier to configure, thanks to the added options for creating child pages, editing, and hiding specific navigation items. In addition, it is possible to remove any item from the navigation (for example, Terms in the Glossary section).

Custom Search Filters

You can now add additional filters to the Catalog Item Search configuration. For more information, see configuring-catalog-item-search.adoc.

Multi-select Option for Dropdown Attributes

We have introduced a new type of entity property. In addition to the single-select property for dropdown attributes, a multi-select property is now available.

UX Improvements

The 13.2.0 release comes with the following UX updates:

  • Single User Assignment

    • When managing access to entities, in addition to assigning permissions to roles, it is now possible to grant permissions to specific users.

  • Data Catalog Search

    • Hits are now highlighted in search results.

    • User sorting is now available.

  • Connection Browser Search

    • You can now search catalog items when browsing a specific connection in a data source.

  • Creating Terms from Context

    • New terms can now be created while assigning terms to a catalog item from any navigation section. If a term you are looking for cannot be found, an option to create it instead appears.

  • Creating Rules From Context

    • You can create a new rule from the profiling results such as Mask analysis or distinct values in Frequency analysis from the Profile and DQ Insights tab.

  • Date/Time Picker

    • We have introduced a component for selecting date/time values.

  • Duplicating Entities

    • Entities can be now easily duplicated, which is especially useful when creating multiple similar entities like terms or DQ rules.

  • Improved Pagination

    • The number of items displayed on one page can now be easily changed.

    • It is now possible to jump to a specific page by specifying the page number.

  • Viewing Anomalies Tab

    • Detected anomalies details can be now also found on the Anomaly Overview page which shows aggregated anomalies for all Catalog Items in which they have been detected.

  • Improved Lookups Management

    • When creating lookup items, you can now use more advanced configuration options (such as defining supported characters and removing duplicates). It is also possible to preview the catalog item data from which you intend to build the lookup. In addition, you can now schedule automatic rebuilds of lookup items using the new Scheduler feature. For more information, see Lookup Items.

    • Lookups can be also built automatically while defining rules, using the value is from catalog item feature. For more information, see Use Lookups in Rules.

Metadata Registry Improvements

Relationships Available for All Entity Types

All entity types are now visible on the Relationships canvas.

Referencing Terms in Rich Text Field

Ataccama ONE Platform now allows for placing terms inline into non-structured text. The functionality will be extended in future releases to accomodate for other entity types. To reference metadata in the rich text fields available in the web application, use a forward slash (/) and select the required term from the list provided.

Data and Metadata Processing

Scheduling

You can now configure scheduling from the UI for the following entities: sources, lookups, terms DQ. The overview of all scheduled events can be found in the Processing Center under Scheduled Jobs.

Salesforce Connector

We have added a native Salesforce connector for reading metadata and data from Salesforce applications, which allows you to quickly document your Salesforce system.

Locking Tables

The option to enable table locking is now available and can currently be used for PostgreSQL and MSSQL data sources.

Reusing Additional Parameters from a Connection String

Keyspaces are now used in CassandraDB connection strings. This can also be applied to other data sources that use databases or schemas.

Stability Improvements

The platform performance has been improved in the following ways:

  • AI Core:

    • The Neighbors microservice now consumes approximately 3 times less memory.

    • The Recommender microservice now provides recommendations approximately twice faster, thanks to the improved performance of Neighbors.

    • All microservices use 80 MB les memory (minimum 70 MB RSS for a microservice without any logic).

  • Data Processing Engine (DPE):

    • Fixed performance issues caused by importing large Snowflake databases.

    • Importing a complete database schema now bypasses JDBC calls and optimizes bulk SQL queries for some data sources (currently, PostgreSQL, Oracle, MSSQL). For large databases, this significantly reduces the importing time (from hours to minutes), depending on the total number of tables.

  • Data Processing Module (DPM):

    • DPEs that have been inactive for a long time are no longer considered when assessing all engine capabilities, which means that jobs requiring the specific capabilities of an inactive DPE are immediately rejected. This improves the platform responsiveness in such instances.

  • Metadata Management Module (MMM):

    • Anomaly detection now takes 10 times less time thanks for improved data fetching.

    • The number of processed events in the Catalog Search plugin has been reduced (removal of duplicated events).

    • Added a dedicated threadpool for flow events (plugin.flow.ataccama.one.async.* configuration properties).

    • The scheduling pool size can now be increased to help prevent delayed scheduled jobs blocking other tasks (property spring.task.scheduling.pool.size).

Observability Improvements

Starting from version 13.2.0, using the previously optional monitoring stack is now mandatory. For monitoring purposes, each ONE module exposes endpoints for readiness and liveness compatible with the corresponding Kubernetes probes, as well as the metrics endpoints for Prometheus. For more information about the configuration and the module-specific metrics, see Monitoring Configuration and Monitoring Metrics.

The monitoring stack consists of Prometheus, Prometheus Node Exporter, and PostgreSQL Server Exporter. In addition, it is possible to use Grafana for data visualization (optional).

The following changes have been made in order to provide more insight into the platform’s functioning as well as help users optimize system resources:

  • AI Core:

    • Added separate metrics for allocated and available attributes in the Neighbors microservice.

  • Data Processing Module (DPM):

    • Added metrics for the number of registered engines in each state.

    • Added metrics for the duration of the engine status check.

    • Added metrics for jobs in DPM in each state, including failed.

  • Metadata Management Module (MMM):

    • Added metrics for the following:

      • Application thread pools

      • Communication between MMM and DPE

      • Job statuses and durations

      • External events

      • gRPC communication

      • Active users

      • GraphQL calls

      • Metadata import

      • Profiling

    • Improved logging for profiling and metadata import.

Was this page useful?