ONE Desktop ONE DQ&C ONE MDM ONE RDM ONE Runtime Server
latest (15.4.0) 15.3.0 15.2.0 15.1.0 14.5.x 13.9.x 13.8.x 13.7.x 13.6.x 13.5.0 13.4.x 13.3.x 13.2.x
User Community Service Desk Downloads
If you can't find the product or version you're looking for, visit support.ataccama.com/downloads

Ataccama 15.4.0 Release Notes

Release Information

Products

ONE Data Quality & Catalog (including ONE Data and Data Stories), ONE RDM, ONE MDM

Release date

2024-11-06

Downloads

support.ataccama.com/downloads/15.4.0

Security updates

Ataccama Security Advisories

Known Issues

This section lists issues we are currently aware of which are considered significant, and which could impact the functionality of your build. We highly recommend reviewing this section before every upgrade.
Module Issue To be resolved in

MDM

ONE-63423: The AI matching option in the MDM Web App menu is visible regardless of the feature status.

15.4.1

Known Issues Resolved

We report known issues in each release. Below is the list of the issues that have been resolved across the platform for this version.

Module Issue Reported in

MDM

ONE-59667: MDM server fails to start when there is a session in the RECEIVING state (state 1) in the tasks_session table of the eh schema.

15.1.0

MDM

ONE-56889: After the history plugin refactoring, backward compatibility is broken and previously defined plugin settings are ignored.

15.2.0

ONE

Data Observability Dashboards Improvements

This version brings the following usability improvements in Data Observability dashboards:

  • More control over which issues are closed when bulk-closing them in the issues table.

    To filter the issues that should be closed, select the date and issue type, or search for specific catalog items. Next, select Close issues (previously Close all issues).

    For more information, see Data Observability Dashboards, section Close issues.

  • Unified functionality on all tabs in the issues table:

    • On the All Issues tab, you can now search for catalog items, the same way as on other tabs.

    • On the Freshness tab, you can now sort issues by the Detected column, the same way as on other tabs.

Generate Full DQ Evaluation Rule Implementation Using AI

Available in Ataccama Cloud deployments only.

Generate full DQ evaluation rule implementation with Gen AI using natural language.

Go to the Implementation tab of one of your rules and select the Use AI to generate Rule logic and inputs option. Enter a plain text prompt using natural language in the Define the rule field.

Generate full rule implementations using AI

For more information, see Generative AI in ONE.

ONE Desktop

Remote Execution on Selected DPEs

Choose which engine will execute your plan remotely by specifying a DPE label in ONE Desktop.

This is particularly useful when you work with multiple DPE instances, such as hybrid DPEs and standard DPEs, enabling you to direct your plan execution to a specific engine. For example, define which engines can connect to private on-premise data sources or direct demanding or complex jobs to a DPE with more resources available.

To learn more, see Remote Plan Execution, section Remote execution on selected DPEs.

MDM

Native Amazon SQS Streaming Connector

We have introduced native support for Amazon SQS to replace the previous JMS-based implementation. To learn how to configure SQS streaming connector, see mdm-integration:amazon-sqs-connection.adoc.

Task Improvements

We have made several enhancements to simplify task management and improve usability:

  • Roles with edit permissions can now modify task names, descriptions, and severity levels.

    Task details editing

  • Assignee and Assignee group lists are sorted alphabetically for easier navigation.

  • Custom permissions now allow you to restrict task visibility only to entities that a role has permission to view. To learn more, see Configuring Permissions, section Configuring a custom permission scheme.

  • Task listings in MDM Web App display only one line of description, with full text accessible on hover and searchable using Ctrl+F.

    Task description example

New Matching Step Parallelism Configuration

We have improved matching step parallelism configuration. Newly introduced properties allow you to configure the number of threads at each level (partition, key, and rule) and set data thresholds to optimize parallel processing. To learn more, see Performance Tuning, section Matching step performance tuning.

History Plugin Persistence

We have introduced database persistence for history event handling. To learn more, see MDM Server Application Properties, section History plugin configuration.

Deprecation Notice

The following features will be deprecated starting from version 16. In case of any questions, contact Ataccama Support for assistance.

Event Publishers

All event publishers, except for the plan publisher, will be deprecated. If you use the deprecated publishers, migrate your existing logic to the plan publisher.

MS SQL and Oracle RDBMS

No further functionalities or optimizations will be developed for Microsoft SQL Server and Oracle RDBMS. Current setups will remain functional, however, we recommend migrating to PostgreSQL. Alternatively, you can switch to Custom Ataccama Cloud, where we manage PostgreSQL on your behalf.

Draft to Task Migration

Automatic drafts to tasks migration will be removed. In case you are upgrading from version 13 or earlier to version 16, you need to first upgrade to version 15.4 LTS.

RDM

Export Records with Published Changes

You can now export records with published changes both through the RDM Web App and RDM REST API and use them for auditing purposes outside of RDM.

The export is available in the JSON format and contains information about the record primary keys, its identifier in the table, as well as the changed columns. This allows you to easily identify each record in a unique manner and see which values were modified, when, and by which user.

To export records from the web application, open the Change Log tab and narrow down your record selection using filters. Select View and then Export. The exported JSON file is then downloaded to your machine. For details, see RDM Change Log.

In addition, the Change Log tab now clearly shows whether any filters are currently applied or whether you are viewing the full changelog. This way, you can quickly make sure you are exporting the records you are interested in, as the filters used also determine which records are included in the export file. You can find this information above the record listing.

Export RDM Changelog

To export the changelog using the API, send a GET or POST request to the new /changelog endpoint. You can filter the records through query string parameters or the request body, respectively. To learn more, see RDM REST API, section Get records with published changes.

New Prometheus metrics

We have made it easier to keep track of the application state using the new ataccama_one_rdm_application_state metric. The metric reports the current status of the application and can be used together with the ataccama_one_business_ready metric to check whether the application can be used or not.

In addition, you can now find out how many HTTP requests the application received and how many users have edit permissions.

For more details, see RDM Monitoring Metrics.

Fixes

ONE

Click here to expand

  • ONE-58423: Only published entities can now be referenced in the Referenced Object property type.

  • ONE-60201: Improved error messaging in Data Stories in cases where you don’t have access to a visualization. The application now clearly indicates whether the issue is due to insufficient permissions or because the visualization contains protected attributes that you cannot access.

  • ONE-60202: In Data Stories, visualization names are correctly displayed on dashboards even if data protection classification is applied to an attribute used in the visualization.

  • ONE-56934: In Data Stories, it was possible to create a new visualization in a collection without specifying a collection name. The Create a new visualization option is now not available until a collection name is provided, preventing you from creating unnamed collections.

  • ONE-62095: Users with view access (view data, view metadata, view page) can no longer edit task type.

  • ONE-61460: Fixed the issue with the catalog item Data Quality tab failing to load if attribute instead of searchAttribute was used for searchConfiguration.

  • ONE-60757: The Add policy condition option is removed from the Create policy screen.

  • ONE-60652: When creating a source, minimizing the Credentials element on the Create Source screen no longer results in an error.

  • ONE-60391: Fixed the issue on the Catalog Items screen where, after searching, it was not possible to select all filtered catalog items using the checkbox.

  • ONE-59938: The AI Term suggestions configuration log widget is removed from the source Overview tab.

  • ONE-59936: Added icons for sources and schemas in the breadcrumbs on the catalog item Overview tab (specifically, Source in the Summary widget).

  • ONE-59626: Fixed the issue where tasks on kanban board showed a progress spinner constantly if the business terms referenced in tasks were deleted.

  • ONE-59300: Users without admin permissions can no longer edit the review workflow.

  • ONE-59177: An error message no longer appears when you try to add relations to an unpublished term. Now it is not possible to add relations before publishing.

  • ONE-59104: Values in the Global search order column on the Search Configurations screen are aligned to the left.

  • ONE-58987: When adding comments to tasks, uploading images to the rich text editor in the task Comments tab doesn’t freeze the sidebar.

  • ONE-58687: When searching on the Terms > Published tab, you correctly see the terms that have been shared with you even if they have no stewardship set.

  • ONE-58647: In Data Observability dashboards Profile Inspector, you can resize the left panel.

  • ONE-58138: The search now accepts queries of at least two characters. Previously, a query had to have at least four characters.

  • ONE-56616: Terms created from a sidebar cannot be published without going through the draft status first.

  • ONE-50515: Removed the incorrect option to create SQL catalog items from a folder on the source Overview tab, in the Workspace widget.

  • ONE-62229: In Data Observability dashboards, the Close issues option on the Terms tab works as expected.

  • ONE-61776: Fixed the issue where it was not possible to see monitoring project results because the project was stuck in the running state in the UI, despite finishing successfully in DPM. The issue was caused by adding the same catalog item twice to the same monitoring project, which is no longer possible.

  • ONE-61556: Old lookup files in ONE Object Storage (MiniIO) are cleaned upon publishing a new version, preventing issues with storage space.

  • ONE-61348: In Data Observability, fixed the time format of Max duration and Expected max time elapsed properties on the detail screen of the freshness issues.

  • ONE-61058: Import preview in ONE Data works correctly when importing CSV files with multi-line records.

  • ONE-60949: The Cron expression in the list of scheduled jobs in the Processing Center now contains the timezone.

  • ONE-60320: The notification threshold is displayed correctly when using values outside the threshold.

  • ONE-60318: Notifications are not sent when the notification threshold is set to 'less than 100%' and the data quality is 100%.

  • ONE-60131: Fixed the issue in monitoring projects where, after deleting a project that had a retention policy enabled in the advanced settings, the UI stayed in the delete draft state, despite the project being correctly deleted.

  • ONE-60029: Removed the Share option from the ribbon menu that appears on the Rule suggestions screen after selecting multiple items.

  • ONE-60028: On the Rule suggestions screen, the checkbox for selecting all catalog item attributes is correctly displayed.

  • ONE-59924: DQ evaluation doesn’t fail on the attribute level when a rule with a parameter of the List of masks data type is applied to a catalog item from Snowflake with pushdown processing enabled.

  • ONE-59817: Data Observability now detects schema changes for BiqQuery data source.

  • ONE-59761: The option to test the rule works as expected.

  • ONE-59727: Selecting Advanced settings on a monitoring project after adding catalog items and publishing doesn’t result in an error.

  • ONE-59721: In Data Observability, you can choose more than one schema in the Set schemas dialog for BigQuery data sources.

  • ONE-59595: Improved error messaging when attempting to save an invalid time series configuration.

  • ONE-59493: Fixed the issue where rule updates were propagated to the monitoring projects before updating the project, leading to the monitoring run failure.

  • ONE-59482: Fixed the issue where after renaming rule inputs, a random version of the rule was propagated to the monitoring project, leading to the monitoring run failure.

  • ONE-59252: Added a missing Refresh option to the catalog item File import settings dialog.

  • ONE-59132: Fixed the issue in transformation plans where the preview was failing if the plan contained a Database output step appending to the catalog item that was used as plan input.

  • ONE-59088: Users with View metadata access can see the DQ Firewalls Documentation tab.

  • ONE-58506: When creating a DQ rule from attribute, the NaN value is not added to new rows in Test rule.

  • ONE-58323: The tooltip for the Next button on the Load to ONE Data screen shows the correct text for catalog items without any applied rules.

  • ONE-58316: Invalid samples are no longer missing after running DQ evaluation on a catalog item from a Snowflake source with pushdown enabled.

  • ONE-57802: In Data Observability configuration, the Set schemas dialog shows schemas for BigQuery data source. Previously, you could only see the whole database.

  • ONE-56680: Fixed the issue in transformation plans where it was not possible to delete columns in ONE Data Writer step, or edit the step configuration to 'Write all attributes'.

  • ONE-56660: Scheduled jobs are executed on the correct day even when using non-English UI.

  • ONE-56481: Attributes on the Rule suggestions screen are listed in the standard attribute order.

  • ONE-55646: Fixed the issue where it was not possible to display a monitoring project in the delete draft status if it referenced a deleted catalog item. Now, scheduled jobs on such catalog items are ignored and you are informed of the issue in the UI.

  • ONE-54385: In transformation plans, the Database Writer step doesn’t produce a validation error when writing a column to catalog item attribute of compatible but not the same data type.

  • ONE-50578: Fixed the issue where it was possible to open data preview in embedded transformation plans, and such preview job was running indefinitely without an option to cancel it. Now, data preview is not available in the embedded transformation plans.

  • ONE-62788: PostgreSQL driver updated to 42.6.2.

  • ONE-62506: Snowflake pushdown jobs cancelled in DPM are now visible as stopped in Snowflake.

  • ONE-62226: DPM logs now include job routing issues.

  • ONE-61872: Null messages are parsed as empty strings in the Kafka Reader step.

  • ONE-61803: Limited the length of columns to prevent exceeding the max blob size of 16 MB in Snowflake pushdown.

  • ONE-61633: Snowflake pushdown domain detection supplement job can be turned off.

  • ONE-60928: In anomaly detection, a backslash (\) in values processed by pattern domain analysis is no longer interpreted as escape.

  • ONE-60539: The JDBC Reader step can connect to Azure Service Principal.

  • ONE-60065: The DPE and DPM configuration property for excluding tables is correctly applied for Snowflake sources.

  • ONE-60032: Added a tooltip to the DPE label field.

  • ONE-60007, ONE-59722: Secret management service works as expected in write credentials.

  • ONE-59816: Salesforce credentials are correctly fetched by the Key Vault.

  • ONE-59397: Sample profiling works as expected for BigQuery Views.

  • ONE-59170: Improved compatibility issues between the runtime engine and Java 8.

  • ONE-58986: Profiling on Databricks partitions works as expected.

  • ONE-58963: Azure SQL connections correctly work with Managed Identity.

  • ONE-58946: In ONE Desktop, reimporting exported runtime configuration imports driver properties as well.

  • ONE-58718: It is possible to create Databricks tables from the JDBC Writer step.

  • ONE-58176: Snowflake pushdown profiling doesn’t exclude null values in pattern and mask analysis.

  • ONE-58144: During metadata import, Snowflake doesn’t query primary and foreign keys for the whole database if a schema name contains an underscore.

  • ONE-57845: Snowflake pushdown DQ evaluation doesn’t fail when attribute names contain spaces.

  • ONE-56469: Improved validations for testing the source connection.

  • ONE-56209: Fixed the issue with integer overflow so profiling results no longer display negative values.

  • ONE-53522: Metadata import doesn’t fail when the schema or selected tables contain a broken view.

MDM

Click here to expand

  • ONE-60306: Fixed the issue where users with custom permissions could not edit or merge records in specific workflow steps.

  • ONE-60668: Task assignment behaves consistently whether performed through the API or MDM Web App.

  • ONE-60582: Manually matched party data doesn’t get overwritten by automated matches during delta loads.

  • ONE-60267: It is now possible to merge instance records manually.

  • ONE-62252: MDM history export works as expected.

  • ONE-59484: It is now possible to profile MDM datasets.

  • ONE-51170: You can create a blank MDM project from the ONE Desktop template without issues.

  • ONE-59368: The value provider for virtual instance entities works as expected when using the custom permission scheme.

  • ONE-59367: The value provider for the workflow step works as expected when using the custom permission scheme.

  • ONE-59168: Selecting a value in the compare view works as expected.

  • ONE-61519: Partial rematch no longer fails.

  • ONE-62182, ONE-59226: Fixed several MDM Web App UI issues.

  • ONE-58591, ONE-60257, ONE-60004: Fixed performance issues with Oracle databases.

RDM

Click here to expand

  • ONE-61834: You can now work with views that contain an MN-reference column without issues.

  • ONE-60321: Fixed the issue with advanced filtering on tables that was causing an error message to appear after logging in.

  • ONE-55364, ONE-61470, ONE-54719: In the Cloud Portal, changes made to the RDM Server configuration are correctly synchronized and loaded.

  • ONE-37767: Improved internal authentication between RDM and Data Processing Engine (DPE).

  • ONE-61598: Using an empty value as a template for filtering no longer leads to issues with filter settings.

  • ONE-61504: When browsing RDM source metadata in ONE, you can open the column preview as expected.

  • ONE-61101: It is now possible to profile columns of LONG type in ONE.

  • ONE-60370: The Enrichment feature works as expected with MN-reference columns.

  • ONE-59723: On the Change Log tab, you can filter by user and table at the same time, without the user filter getting removed.

  • ONE-59201: The Edit children recursively option in the web application is correctly selected or cleared according to the configuration settings.

Was this page useful?