ONE Desktop ONE DQG ONE MDM ONE RDM ONE Runtime Server
latest (15.2.0) 15.1.0 14.5.x 13.9.x 13.8.x 13.7.x 13.6.x 13.5.0 13.4.x 13.3.x 13.2.x
User Community Service Desk Downloads
If you can't find the product or version you're looking for, visit support.ataccama.com/downloads

Ataccama 13.2.0 Release Notes

This page contains the release notes for the release of version 13.2.0 of the entire Ataccama ONE Gen2 platform. Release information affecting all modules in the platform can be found in the first section, whereas platform and module specific information can be found in the ONE, MDM and RDM sections.

Release Information

Products

ONE Data Governance, ONE Data Quality and Governance, ONE RDM, ONE MDM

Release date

2021-07-14

Downloads

support.ataccama.com/downloads/13.2.x

Known Issues

This section lists issues we are currently aware of which are considered significant, and which could impact the functionality of your build. It is highly recommended to review the Known issues section before every upgrade.
Module Issue To be resolved in

ONE Desktop

ONE-23846: In some cases when working on ONE plans the server validation may expire after longer periods of time and users are not able to run remote execution etc.

Workaround: Disconnect and reconnect to the ONE Platform server to dispose of authentication tokens and create them again.

13.3.0

ONE Web Application

ONE-24055: It is possible to create new property filters without defining the target Property. This will not function, but validations are not working so no error is thrown.

13.3.0

RDM

ONE-24105: It is not possible to upgrade Oracle-based projects from version 12.x to 13.0-13.2 due to incompatibility in column naming.

13.3.0

RDM

ONE-27130: After upgrade, RDM workflows are by default disabled and need to be enabled manually.

13.6.0

MDM

ONE-24177: Parts of MDM Webapp will not work correctly when app server context path is used.

13.3.0

MDM

ONE-25239: MDM History Event Handler fails.

Workaround: Before migration or upgrade, all events have to be committed and event storage has to be empty.

13.6.0

ONE Web Application

ONE-24621: Edit button missing for manually uploaded lookup files.

13.3.0

ONE Web Application

ONE-24745: Unable to add more than 10 catalog items to a monitoring project.

13.3.0

Known Issues Resolved

We report known issues in each release. Below is the list of the issues that have been resolved across the platform for this version.

Module Issue Reported in

ONE Web Application

ONE-23266: It is not possible to create lookup items from virtual catalog items (VCIs).

13.0.0, 13.1.0

MDM

ONE-11568: Some features of turning off overrides regarding exports and life cycles are yet to be implemented.

13.1.0

RDM

ONE-22844: DATE datatype import/export generated as STRING.

13.1.0

RDM

ONE-22656: DATE datatype cannot be used for validation.

13.1.0

RDM

ONE-22655: DATE datatype cannot be used for enrichment.

13.1.0

ONE

If you wish to upgrade from version 13.1.0 to version 13.2.0 of the Ataccama Data Quality and Data Governance suites, contact your account manager to discuss details.

We now support and recommend using Keycloak 13.0.1 version. For a full list of supported dependency versions, see supported-third-party-components.adoc.

ONE Desktop

Support for Salesforce and SAP Servers

We have added support for Salesforce and SAP RFC systems, allowing users to work with Salesforce and SAP data, as well as use preconfigured SAP functions. For more information, see one-desktop:server:connect-to-a-server.adoc, sections Salesforce and SAP RFC respectively.

To be able to consume and modify data from these applications, we have introduced the following reader and writer steps:

  • Salesforce Reader: Reads data from user’s Salesforce application.

  • Salesforce Writer: Allows users to update or delete data in Salesforce SObjects.

  • SAP RFC Reader: Reads data from user’s SAP RFC application.

  • SAP RFC Execute: Allows users to execute SAP functions on SAP RFC data.

ONE Platform Browser Authentication

From ONE 13.2.0, you can use browser authentication for connecting to ONE Platform server.

To connect to ONE Platform using OpenID web browser authentication follow these steps:

  1. Select File > New > Server.

  2. In the New Server dialog box, select OpenID Connect via Web Browser in Type.

    ataccama 13.2.0 release notes register new server

  3. Save connection and connect.

  4. You will be prompted to allow web browser authentication.

    ataccama 13.2.0 release notes browser authentication

  5. Log in using your credentials.

    ataccama 13.2.0 release notes browser authentication credentials

  6. After successful authentication you can close the browser tab and return to ONE Desktop.

Support for Nested Components

In 13.1.0 we introduced nested transformation components. A nested component is a transformational component that is used inside another component, which can then be included in complex plans. For example, this lets you cleanse or transform data in virtual catalog items using one of preconfigured components, without modifying the source data. To learn more, see Nested Components.

Your nested components can now be easily imported and converted into ONE components by dragging them from File Explorer into ONE Explorer. The component file and all linked files will be imported into the componentFiles folder on your ONE Platform Server. All paths in the component file are converted to relative paths so that they work independently of your local naming conventions.

To import a nested component into ONE Platform follow these steps:

  1. Drag a component from File Explorer to any ONE component in ONE Explorer.

    ataccama 13.2.0 release notes one explorer

  2. If there are any external files that the imported component refers to they will be imported too. All paths to external files are modified during import so that the function of the component is intact.

    ataccama 13.2.0 release notes import files

You can also import nested components by dragging them directly onto a ONE component canvas area. To do this:

  1. In ONE Explorer, double-click the ONE component that you want to edit in the canvas.

  2. Drag and drop any files and components that you want to nest in the component and import to the ONE Platform Server.

  3. When you are ready, save your changes, right-click your ONE component and click Publish to apply changes to the server.

    Conversion of paths

    When you drag and drop a file or component onto ONE component canvas the file paths will stay local until you publish the changes to the server. All paths in the component will be converted to relative during publishing of the component.

    ataccama 13.2.0 release notes conversion of paths

Configure Components in ONE Desktop

Now you can configure the state and the main file for ONE components in ONE Desktop:

  1. Right-click a ONE component and select Configure.

  2. You can edit Implementation state and select the main file in order to determine which components are nested.

    ataccama 13.2.0 release notes configure components

Enhanced US Address Identifier Step

We have enhanced the function and interface of the US Address Identifier step making sure no data is omitted from the output. Make sure you have the latest lookup files for correct functioning of the Ataccama ONE Platform.

In 13.2.0 we have:

  • Added a new Cleansed tab and added new fields to the Output tab.

  • Reworked the part of validation logic responsible for parsing of different parts of the address, e.g. city, state, units, PO boxes etc.

  • Added a new lookup file to improve unit information parsing.

  • The step supports only 9-digit ZIP codes, so ZIP codes are now trimmed accordingly. 9-digit ZIP codes consist of a State code and ZIP code.

  • Added street addresses to the output for addresses that have been identified as PO BOX type (only the PO BOX part is validated and street address is not).

  • Added new explanation codes related to all recent changes.

ONE Runtime Server

We have added a new Notification Handler Server component, thanks to which users can now schedule running a script or a workflow based on certain changes in Metadata Management Module (MMM). As such, the component is similar to the Scheduler Server Component, with the key difference being how schedules are initiated. The component is integrated in ONE Runtime Server Admin (Notifications tab in Workflows), where you can see the configured notification handlers, keep track of the job execution, as well as unsubscribe the handlers that you no longer need. Information about MMM events can be passed through variables to both script and workflow jobs. For more information about the component, see Notification Handler Component. For more information about how to configure and use notification handlers for different types of jobs, see Notifications Handler.

ONE Web Application

Creating Rules from Analyses

It is now possible to create a rule from Mask, Pattern, and Frequency analysis of an attribute, using the Profile & DQ Insights tab. Rules can be created for the purpose of identifying either correct or incorrect behavior:

  • To create a rule of the correct mask, pattern, or frequency, the matches mask condition is introduced.

  • To create a rule of the incorrect mask, pattern, or frequency, the not matches mask condition is introduced.

Scheduler for Events

In a number of places in the application, it is now possible to set up a schedule for certain actions. These will then be triggered automatically by the application at the specified time.

Currently the scheduler can:

  • Run monitoring projects.

  • Rebuild lookup items.

  • Run the Document, Discover, or Import flow on a data source.

After opening the scheduler from one of these locations, the configuration sidebar opens. This will show existing scheduled events, if any, or you can select Add Scheduled Event.

ataccama 13.2.0 release notes schedule event

The timing for the event can be set in two ways:

  • The Basic view of the Scheduler for simpler requirements. Define the schedule via the user interface.

  • The Advanced view of the Scheduler. Define the schedule using Cron format.

Scheduled events can be seen in the Processing Center under Scheduled Jobs.

ataccama 13.2.0 release notes scheduled events list

Managing Top-level Default Access for Entities

With the top-level configuration of any array embedded entity (such as sources, rules, terms), it is possible to set the default permissions, which are propagated to all root entities (to all sources, all rules, all terms). When the default permissions are set on the top-level entities, all changes will be propagated similarly on all available entities in the list.

Viewing Anomalies Tab

Detected anomalies details can be now also found on the Anomaly Overview page which shows aggregated anomalies for all Catalog Items in which they have been detected.

ataccama 13.2.0 release notes anomaly overview

Lookup Improvements

When creating lookup items, you have more advanced configuration options (such as defining supported characters and removing duplicates) and the ability to preview the data of the catalog item from which you intend to build the lookup. You can also now schedule automatic rebuilds of lookup items using the new Scheduler feature. For more information, see Lookup Items.

Build lookups from rules

You can now build lookups automatically during rule definition, using the value is from catalog item feature. For more information, see Use Lookups in Rules.

Ability to Reference Metadata in Rich Text Field

In the rich text fields available in the web application it is now possible to reference metadata, using a forward slash (/) and selecting the required term from the list provided.

Web App Configuration

Available for power users only.

Custom Search Filters

You can now add additional filters to the Catalog Item Search configuration. For more information, see configure-catalog-item-search.adoc.

ataccama 13.2.0 release notes custom search filters
Detailed Lineage Update

A Manta connection link can be now set to enable direct connection to the Manta graph from the Lineage tab.

The lineage can be also calculated on all available data sources by clicking the Calculate Lineage button on the Sources page.

List of Values Update

It is now possible to assign several values from the list of options. This can be done by creating another entity in Metadata Model serving as a target.

Hide Left Navigation Menu

It is now possible to hide pages from the left-side navigation menu in the case that they should not be seen by the users, but deleting them is undesirable.

Upgrade Center Changes

The Upgrade Center modes were updated and validation checks were added to verify the state of the application before introducing changes. The validation fails can be located and the particular errors identified.

Inconsistencies between roles in Keycloak and their permissions that are identified during the upgrade procedures are now shown on the Root role mapping tab of the Permissions Model page.

Changes in Default Permissions Configuration

In the default permissions configuration, a new capability has been added, governance-editor, which will take the place of the governance-admin capability for the Keycloak role MMM_data-governor. This means that in the future the data-governor role will not have policy enforcement rights. For more information, see user-access-management:manage-permissions-users-roles.adoc.

AI Core

In this version, we have made the following changes:

  • There are now two possibilities for upgrading AI Core microservices: manually, by running a Kubernetes job for on-premise deployment, or by restarting the newly added Upgrade microservice (must be enabled first).

  • The Neighbors microservice now uses fixed memory allocation instead of increasing the number of fingerprints as needed. The default maximum number of attributes (1M) can be adjusted for smaller deployments to save RAM. As a result, it is possible to configure a fixed Kubernetes memory limit that is never overstepped. Furthermore, the overall RAM consumption of the Neighbors microservice has been reduced approximately 5 times thanks to a different algorithm applied. The Kubernetes memory limits can therefore be downsized accordingly.

  • Anomaly Detection now supports more generic input built around the category entity.

  • We have added a script for querying the microservice status. The script, located in the build package (ai/bin/healthcheck.sh), produces a human-friendly output containing information about the microservice’s health status and the status of its connection to the dependencies, such as the database and the authentication provider. As the script uses the configuration already provided for the microservice, it can be run directly from the command line without any added arguments.

MDM

Record Labels Update

Record template labels are now generated into mda-label.gen.properties and defined using a more versatile syntax allowing for DQC expressions. Old definitions can be upgraded using an upgrade template. See Constructing Record Detail Templates.

ataccama 13.2.0 release notes records labels update

Salesforce Stream Consumer

Salesforce can now be used as a stream consumer source. See mdm:input-and-output-interfaces/configuring-stream-consumers.adoc.

Upgrade notes

To determine your upgrade path for version 13.2.0, please see the MDM Upgrade Procedure and MDM 13.2 Upgrade Notes.

RDM

Keycloak Synchronization

There are now several new ways to synchronize roles and users with Keycloak:

  1. Automatic sync on user login.

  2. Manual sync using a button on the Permissions tab. See setting-permissions-in-rdm.adoc.

  3. Scheduled sync using a definition in application.properties. See rdm-application-properties.adoc.

DATE Datatype

The new DATE datatype has been further improved and it is now compatible with the DQC DAY datatype.

Upgrade Notes

There are no version specific upgrade notes for 13.2.0, to determine your upgrade path see RDM Upgrade Procedure.

Fixes

ONE

ONE Desktop

Click here to expand

  • ONE-22549: Server names are not saved in plan metadata anymore. Paths in metadata are now converted to relative paths.

  • ONE-22664: File paths are now converted to relative when you import a component into ONE Platform and convert it to a ONE component.

  • ONE-22731: Components and nested components can now be easily dragged and dropped from File Explorer directly onto your canvas or into ONE Explorer in order to import them.

  • ONE-22533: It is now possible to connect to a server using Keycloak OpenID browser authentication.

  • ONE-23609: You can edit properties of ONE components in ONE Explorer.

  • ONE-23797: We have changed all paths in ONE components to relative.

  • ONE-19142, ONE-21919: We now can identify counties in US Address Identifier step.

  • ONE-23064: Treatment of addresses identified as PO BOX was improved to prevent any issues of data loss.

  • ONE-21090: Records with missing state information are now enriched. 9-digit ZIP codes are now returned for all the addresses that were validated up to BUILDING level.

  • ONE-19031: Support for SAP RFC data source.

  • ONE-20746: Support for Salesforce data source

  • ONE-22837: If there’s a CI attribute of a data type for which there’s no icon (JSON data type) the graph crashes.

  • ONE-23771: The profile results are saved in MinIO forever without any retention.

  • ONE-22362: Profiling table with multiple partitions fails.

  • ONE-23569: When profiling a virtual catalog item terms were applied automatically on attributes as expected, however, parent terms were not tagged on the catalog item as a whole.

  • ONE-23480: It is not possible to add more than 10 rules in the Monitoring Report.

  • ONE-22864: Cannot create the new workflow because of the validations.

  • ONE-23363: Unable to configure mTLS for API calls from plans (for example, using JSON Call step).

ONE Runtime Server

Click here to expand

  • ONE-21390: Event-trigger .EWF workflow runner.

  • ONE-21340: Configuration of logging is not possible via web environment of ONE Runtime Server Admin because the Logging Setup page has been deprecated. Logging can only be configured using configuration files.

ONE Web Application

Click here to expand

  • ONE-22368: Direct link to Manta can be configured from the Lineage page.

  • ONE-21399: Multiple values can be assigned from the list.

  • ONE-22367: Relation Graph shows the custom entities.

  • ONE-23333: It is now possible to schedule Documentation Flow runs from the Sources connection.

  • ONE-22782: The Anomalies Overview page was added, which aggregates all catalog items with anomalies in one place.

  • ONE-23683: The link renderers are configurable and validated from Debagging Tools.

  • ONE-23526, ONE-23946: Upgrade Center Validations are introduced to verify the application status and search for issues and incompatibilities before the upgrade.

  • ONE-23549: With the top-level permissions setting on the nodes it is now possible to edit the default permissions on the high-level nodes (terms, sources, rules).

  • ONE-22889: It is now possible to schedule the runs of the monitoring projects.

  • ONE-22890: Scheduled events can be viewed in Processing Center.

  • ONE-23158: It is now possible to schedule events on terms.

  • ONE-21622: It is possible to create rules directly from mask, frequency, and pattern analysis in Catalog Items profiling results.

  • ONE-21582: Catalog Event API.

  • ONE-22964: Rules with dateAdd function fail in Spark jobs.

  • ONE-23654: Invalid samples sometimes display float numbers in scientific notation.

AI Core

Click here to expand

  • ONE-22615: Upgrade microservice for running DB upgrades.

  • ONE-23329: Limit for neighbors fingerprint index size.

  • ONE-19661: Pre-allocate memory for fingerprints in neighbors on start.

  • ONE-19075: Generic API for Anomaly detection implemented, new structure of categories, optimization of code.

Data Processing Module and Data Processing Engine

Click here to expand

  • ONE-21484, ONE-16106, ONE-17097: Reworked engine registry with more resilient status checking and new configuration parameters.

  • ONE-22552: Improved the efficiency of the internal job queue.

  • ONE-21548: Run results not being persisted.

  • ONE-22376: DPE (cloud deployment) does not start after graceful shutdown.

ONE Spark DPE

Click here to expand

  • ONE-22183: ONE Spark DPE is now able to use Azure Managed Identity Services to authenticate using Azure KeyVault directly.

  • ONE-22324: It is now possible to connect to your Databricks cluster using Azure Active Directory access token.

  • ONE-20787: Issues with correct job routing.

  • ONE-23012: We have added a new configuration parameter that determines automatic mounting of Databricks. Set #plugin.executor-launch-model.ataccama.one.launch-type-properties.SPARK.dbr.autoMount to false to disable automatic mounting.

MDM

Click here to expand

  • ONE-24093: Editing record details with lookups throws error.

  • ONE-23985: Comments not stored in database.

  • ONE-23057: Output columns on read step.

  • ONE-22801: Salesforce stream consumer support.

  • ONE-22779: Case sensitive parameter.

  • ONE-22735: UI translation into Russian.

  • ONE-21981: Override icon not displayed on detail screen.

  • ONE-21933: Issues with high number values.

  • ONE-21761: Escaping of the # character in issue description.

  • ONE-12301: New label definition.

  • ONE-12086: Rematch behavior on master detail view.

RDM

Click here to expand

  • ONE-22187: Permission sync issues on multiple instances.

  • ONE-19937: New options for users/roles sync.

  • ONE-11930: Issues with high number values.

Was this page useful?