If you can't find the product or version you're looking for, visit support.ataccama.com/downloads

Ataccama 13.9.0 Release Notes

This page contains the release notes for version 13.9.0 of the Ataccama ONE Gen2 platform. Release information affecting all modules in the platform can be found first, and module specific information can be found in the corresponding ONE, MDM, and RDM sections.

Release Information

Products	ONE Data Governance, ONE Data Quality and Governance, ONE RDM, ONE MDM
Release date	2022-09-13
Downloads	support.ataccama.com/downloads/13.9.x
Security updates	Ataccama Security Advisories

Products

ONE Data Governance, ONE Data Quality and Governance, ONE RDM, ONE MDM

Release date

2022-09-13

Downloads

support.ataccama.com/downloads/13.9.x

Security updates

Ataccama Security Advisories

Known Issues

These issues could impact the functionality of your build; make sure to check them before proceeding with the upgrade.

Module Issue To be resolved in

Module	Issue	To be resolved in
ONE Desktop	ONE-32661: In ONE Desktop, successful attempts to log in through a web browser result in an error page.	14.0
MMM	ONE-37958: In ONE, the Approval Requests screen becomes inaccessible if an embedded entity property (Single Embedded Entity or Array Embedded Entity) is removed from an entity while an instance of that entity is waiting for approval.	13.9.2
ONE	ONE-36847: Data preview not working for "is from catalog item" rules.	14.1
MDM	ONE-37508: `RematchIfChanged` overrides manual matches overrides if configured.	13.9.2, 14.1.0
MDM	ONE-44522: In PostgreSQL-based MDM solutions, a performance slowdown is possible, which might affect draft publish speed.	13.9.4
ONE	ONE-39600: Full-text search cannot be configured on the recursive entities. When configuring search on recursive entities, the search indexation breaks on these entities.	14.3

ONE Desktop

ONE-32661: In ONE Desktop, successful attempts to log in through a web browser result in an error page.

14.0

MMM

ONE-37958: In ONE, the Approval Requests screen becomes inaccessible if an embedded entity property (Single Embedded Entity or Array Embedded Entity) is removed from an entity while an instance of that entity is waiting for approval.

13.9.2

ONE

ONE-36847: Data preview not working for "is from catalog item" rules.

14.1

MDM

ONE-37508: RematchIfChanged overrides manual matches overrides if configured.

13.9.2, 14.1.0

MDM

ONE-44522: In PostgreSQL-based MDM solutions, a performance slowdown is possible, which might affect draft publish speed.

13.9.4

ONE

ONE-39600: Full-text search cannot be configured on the recursive entities. When configuring search on recursive entities, the search indexation breaks on these entities.

14.3

Known Issues Resolved

We report known issues in each release. Below is the list of the issues that have been resolved across the platform for this version.

Module Issue Reported in

Module	Issue	Reported in
ONE Desktop	ONE-31494: Newly created steps can provide DQ results from monitoring projects. These can be utilized where previously the ONE Metadata Reader step would have been (due to changes in DQ storage the Metadata Reader step can no longer be used to access DQ results).	13.5
ONE Desktop	ONE-31495: Newly created steps can provide DQ results from catalog. These can be utilized where previously the ONE Metadata Reader step would have been (due to changes in DQ catalog storage the Metadata Reader step can no longer be used to access DQ results from catalog).	13.8
MDM	ONE-31814: When MDM is started with a blank project, no roles have permissions for the Admin Center due to missing `mda-permissions.gen.xml` file.	13.8
MDM	ONE-31283: OnlineCtl commands for workflow (EWF) do not work in MDM Server.	13.7
ONE	ONE-33115: A metadata import or MANTA processing job remains in the `RUNNING` status and prevents other queued jobs from starting.	13.3.0

ONE Desktop

ONE-31494: Newly created steps can provide DQ results from monitoring projects. These can be utilized where previously the ONE Metadata Reader step would have been (due to changes in DQ storage the Metadata Reader step can no longer be used to access DQ results).

13.5

ONE Desktop

ONE-31495: Newly created steps can provide DQ results from catalog. These can be utilized where previously the ONE Metadata Reader step would have been (due to changes in DQ catalog storage the Metadata Reader step can no longer be used to access DQ results from catalog).

13.8

MDM

ONE-31814: When MDM is started with a blank project, no roles have permissions for the Admin Center due to missing mda-permissions.gen.xml file.

13.8

MDM

ONE-31283: OnlineCtl commands for workflow (EWF) do not work in MDM Server.

13.7

ONE

ONE-33115: A metadata import or MANTA processing job remains in the RUNNING status and prevents other queued jobs from starting.

13.3.0

ONE

Anomaly Detection Improvements (Irregular Timestamps)

Time-dependent anomaly detection within monitoring projects has been improved both in terms of capability and user interaction. The model can now detect irregular or missing timestamps and subsequently exclude them from the calculation or impute suitable values. This makes anomaly detection on time series more accurate. In addition, if no periodicity is specified, the system can derive it from the timestamps.

You now also need to specifically select to run time-dependent anomaly detection, whereas previously this was just assumed when periodicity was provided. Furthermore, you can either provide a periodicity value yourself or leave it to the system to derive it.

DQ Notifications Improvements

In addition to the existing notifications available on monitoring projects, from 13.9.0 it is possible to configure notifications for specific catalog items, groups, or rule instances, based on data quality thresholds you define. As well as the increased granularity, further improvements include being able to notify individual users (not just groups and roles), and being able to use filters within the notification configuration.

Both the existing general notifications (monitoring failed, critical error in structure, new results, DQ issue, or anomalies) and the custom notifications, will be listed in the monitoring projects Notifications tab. From there, you can edit, turn them off or on, or remove them as required.

DQ Rules Included in New Import and Export Capabilities

DQ rules can now be included in export plans and then imported into a new instance. For more information, see Import and Export.

Create Permissions Functionality Extended

In version 13.9 of ONE, we have extended the permissions functionality. Now you can configure the number of users with the create permissions, which allows non-admin users to create new assets in ONE. Create permissions let users create a new asset on the node page using the Create button. For users without the necessary permissions, the button is disabled.

This way, you can separate assets for various groups and ensure that all users have the appropriate access rights. Consider the following when working with new create permissions:

When any user creates an asset, this user automatically becomes the owner of that asset and can then view and edit it. This applies to all users with any role that gives permission to create an asset.
A new Keycloak role MMM_creator is added. Users with this role can create assets of any type. Since this is an extension of the previous model, create capabilities of all other roles remain unchanged and valid. In other words, admin users can still use their existing permissions to create assets of the same type and they can still edit all assets.
The user who created an asset can then share it with any other user or role through the Access tab. Meanwhile, all the Viewer role capabilities remain unchanged and users with the super-viewer role can still see all assets.

The new functionality is turned off by default and must be manually enabled via the JSON configuration file. For more information, see Manage Permissions of Users and Roles, section Configuring custom permissions.

For various MMM_creator role mapping example configurations, see Configuring Permissions Frequent Scenarios.

Global Search

Since 13.9 the new full-text search is enabled on the whole application and is a good starting point for searching data entities and locating the area for the future detailed search if necessary. For more information, see Search. To learn how to configure Global Search, see full-text-search-configuration.adoc.

AWS IAM Authentication for AWS resources

Amazon S3 bucket can now be accessed using AWS IAM roles.

This functionality is currently not available for Ataccama ONE PaaS.

New Lineage Schema

With the new MANTA lineage schema more data sources can be configured for lineage. MANTA-integrated lineage is now available not only for catalog items from Microsoft SQL and PostgreSQL but also for Oracle and Teradata sources.

The new schema must be defined in Data Processing Engine (DPE) configuration via plugin.manta-integration.ataccama.one.manta-integration.url-viewer-gui property before the 13.9 version runs for the first time. If the property is not configured, lineage data is unavailable. For more information, see Lineage Configuration.

Terms Suggestions Synchronization

In versions 13.8 and earlier, Term Suggestions and the Metadata Management Module (MMM) communicated using a term suggestions database, which needed to be configured. Since version 13.9, the connection is enabled via the new Api service and it is no longer necessary to configure the database connection. If you are upgrading from an earlier version, the Api service needs to be enabled for 13.9. For more information, see ONE Gen2 Upgrade Guide.

As the Term Suggestions services have improved in stability, it should not be necessary to initiate Term Suggestions synchronization when working with the application as usual. The only exception to this is when modifying the metadata model, for example, by restoring database from a snapshot or by importing any initial content (such as default or demo).

After restoring the database from a snapshot, Term Suggestions need to be synchronized on the Global Settings > Term Suggestions tab:

ataccama 13.9.0 release notes term suggestions synchronization

New Steps in ONE Desktop to Access DQ Results

The API changes resulting from the changes to DQ project and DQ catalog results storage meant that the ONE Metadata Reader step could no longer be used to access results. This release brings new steps that can be used in addition to the Metadata Reader step (or other inputs) to access DQ results:

DQ Attribute Aggregation Results: Returns results of certain DQ aggregations for attributes.
DQ Catalog Item Aggregation Results: Returns results of DQ aggregations for catalog item.
DQ Monitoring Project Aggregation Results: Returns results of certain aggregations (custom, catalog item level, attribute level).
DQ Monitoring Project Check Results: Returns results of certain DQ checks. Results also include explanations of invalidity.
DQ Monitoring Project Filter Values: Returns data values of filter attributes. These values can change in each processing.
DQ Monitoring Project Results: Returns project validity results aggregated over all catalog items in the monitoring project.
DQ Rule Instance Results: Returns results of DQ rule instances. Results also include explanations of invalidity.
DQ Term Aggregation Results: Returns aggregated DQ results for a whole term.
DQ Term Attribute Aggregation Results: Returns results of term aggregation for an attribute.

Updated getParameterValue Function in ONE Desktop

Starting from 13.9.0, we have changed how getParameterValue function behaves. To retrieve the value of an environment variable, this option must now be explicitly configured for that variable.

To learn more about how to enable this when running a plan or debugging a step, see Commonly Used Functions, section Other operations.

MDM

Date Picker

The date picker in both MDM and RDM now allows easier selection of a particular date.

Loading Additional Drivers

MDM now natively supports the ability to load additional libraries from a dedicated location in both on-premise and PaaS deployments. External driver libraries for the most common data providers and consumers have been tested. See Loading Additional Drivers for details.

AI Matching Improvements

Generating training pairs is now as fast as it was originally before the AI Matching persistence was introduced in version 13.8. In addition, a Remove Model button has been added to the more options menu that allows you to remove an existing AI Matching model and start a new one from scratch.

AI Matching now also supports standard Prometheus metrics for monitoring (see Monitoring Metrics).

Logging Filters and Security Headers

MDM Server HTTP requests can be selectively logged using logging filters and server responses can be configured to contain security headers (see MDM Server Application Properties).

Upgrade Notes

To determine your upgrade path for version 13.9.0, please see the MDM Upgrade Procedure and MDM 13.9.0 Upgrade Notes.

RDM

Date Picker

The date picker in both MDM and RDM now allows easier selection of a particular date.

Upgrade Notes

There are no version specific upgrade notes for 13.9.0, to determine your upgrade path see RDM Upgrade Procedure.

Fixes

ONE

Click here to expand

ONE-31183: You can now connect to the Microsoft OneDrive Storage data source.
ONE-30867: We have significantly improved the preprocessing throughput when reading data. Queries for retrieving the row count and the last partition are now executed in DPE, not DPM. Since DPM no longer needs to wait for these results in the preprocessing phase, the DPM throughput is increased.

Due to this change, DPE might stay longer in the STARTING status compared to earlier versions.
ONE-32141: We have made changes to Cleanse Phone Number US steps in ONE Desktop:
- Cleanse Phone Number US (simple) Step: No longer delivered with the build. Can still be used in existing solutions.
- Cleanse Phone Number US (complex) Step: Now uses a new source file for reference data from NANPA.
ONE-24000: Reader components for profiling jobs are now displayed in the DPM Admin Console.
ONE-29093: Resource allocation statistics remain preserved after restarting DPM.
ONE-30096: In the DPM Admin Console, XML masking no longer affects non-XML values.
ONE-30468: We have improved how files are organized in ONE Object Storage. The new structure for the executor bucket is as follows: executor/$last_3_digits_job_id/$next_last_3_digits_job_id/$job_id-(metadata or results).

Existing files are not affected.
ONE-31392: Fixed CVE-2022-23221 in DPE by upgrading affected third-party software.

ONE-32253: In order to create the directory that holds temporary work files for local jobs, it is no longer necessary to define plugin.executor-launch-model.ataccama.one.launch-type-properties.LOCAL.env.TEMP_ROOT property.

If you’re upgrading from version 13.7.0 and earlier, the temporary directory is created automatically. To configure the path to the temporary directory, use plugin.executor-launch-model.ataccama.one.launch-type-properties.LOCAL.env.TEMP_ROOT property according to the instructions described in DPE Configuration.

If you’re upgrading from version 13.8.x and you have already customized the path to the temporary directory using -Djava.oi.tmpdir as part of the plugin.executor-launch-model.ataccama.one.launch-type-properties.LOCAL.env.JAVA_OPTS property, be aware that it is now the property with the highest priority. In that case, the same path should be provided in plugin.executor-launch-model.ataccama.one.launch-type-properties.LOCAL.env.TEMP_ROOT.

ONE-32337: Issues with DPM and DPE connectivity have been fixed. The connection between DPM and DPE is now easily restored after network failure.

We have lowered the default value of the DPE configuration property ataccama.one.dpe.service.dpm.connection.firewall-friendly.max-connection-inactivity from 10m to 1m. In case you already modified this value, we recommend changing it to match the current default value (1m) to ensure short waiting time before DPE reconnects again.

ONE-31598: In ONE expressions, we disabled using any environment variables except those defined by the ataccama.one.dpe.env.variables.allowed property.
ONE-31475: We fixed the issue with Azure SQL database metadata import failing for some databases.
ONE-31612: When running Document flow on Snowflake, technical tables are now excluded from job results.
ONE-31532: Using isNumber function when running Snowflake pushdown profiling is now working as expected.
ONE-31528: Only relevant schemas are now displayed while browsing a Databricks data source.
ONE-28719: Improved error handling for data source connections in the Data Catalog.
ONE-22771: XML Reader in Virtual Catalog Items now runs as expected on MinIO and Amazon S3 sources.
ONE-29110: In cases where a reverse proxy is used with ONE Runtime server, you can now provide a public URL to make sure the correct URL is used for online services.
ONE-26473: JMS services can now be reloaded from the Admin Center.
ONE-32536: Catalog items can now be deleted from a project without issues.
ONE-23959: Editing DQ rule implementation in the sidebar now works as expected.
ONE-30609: Fixed the discrepancies in chart displays of overall validity.
ONE-30707: Improved the look of the implementation sidebar for rules with is from catalog item condition.
ONE-31717: Anomaly detection jobs no longer fail after changing catalog item attribute data type.
ONE-32045: Changing a rule dimension no longer results in issues with report configuration.
ONE-32197: Structure checks that fail due to missing catalog items now show the correct error.
ONE-30036: Reverting rules to an older version no longer fails when many rule versions need to be verified first.
ONE-30363: Anomaly checks are now correctly restored when reverting from a project in which they have been removed to an older project in which they were enabled.
ONE-31524: Group by section is now present on the implementation tab for aggregation rules in monitoring projects.
ONE-28961: The Group Aggregator step now works as expected when dealing with integers exceeding the maximum representable value. If the end result is larger than this value, an error is shown.

MDM

Click here to expand

ONE-11216: Date Picker improved.
ONE-30291: Progress indicator now appears when searching in MDM Webapp.
ONE-31577: Refresh, Vertical View, and Column Setup buttons added in horizontal compare view.
ONE-31303: Only users with the correct roles can now log in to the webapp (instead of being able to log in but seeing no content due to lack of permissions).
ONE-18394: Text overflowing issue fixed.
ONE-30276: The pagination is now displayed correctly in case the requested page is out of the available range.
ONE-30849: MDM is now able to start if ADLS is added as a data source.
ONE-30278: MDM can now load external drivers not included with the MDM packages.
ONE-30059: ONE Catalog integration functionality fixed.
ONE-31403: Global sampling settings in ONE now apply to virtual instance entities in MDM Webapp.
ONE-31282: Fixed issue with MDM not starting after upgrade due to obsolete jmsService.
ONE-31280: Step connections in validation plans are now preserved after upgrade.
ONE-20884: MDM REST API now preloads records as expected.
ONE-32038: The upgrade template now works as expected.
ONE-29889: AI Matching training pairs generation is now as fast as it was before 13.8. Basic Prometheus metrics are now available for the service. A Remove Model button has been added.
ONE-27336: Added support for an MDM Server logging filter (configured in application.properties).
ONE-27337: Added support for client security headers (configured in application.properties).

RDM

Click here to expand

ONE-30789: RDM now works in Safari browser.
ONE-11216: Date Picker improved.
ONE-29370: Only users with the correct roles can now log in to the webapp (instead of being able to log in but seeing no content due to lack of permissions).
ONE-31840: today() function now works in RDM filters.

Was this page useful?