Ataccama 13.2.0 Release Notes
This page contains the release notes for the release of version 13.2.0 of the entire Ataccama ONE Gen2 platform. Release information affecting all modules in the platform can be found in the first section, whereas platform and module specific information can be found in the ONE, MDM and RDM sections.
Release Information
Products |
ONE Data Governance, ONE Data Quality and Governance, ONE RDM, ONE MDM |
|---|---|
Release date |
2021-07-14 |
Downloads |
Known Issues
| This section lists issues we are currently aware of which are considered significant, and which could impact the functionality of your build. It is highly recommended to review the Known issues section before every upgrade. |
| Module | Issue | To be resolved in |
|---|---|---|
ONE Desktop |
ONE-23846: In some cases when working on ONE plans the server validation may expire after longer periods of time and users are not able to run remote execution etc. Workaround: Disconnect and reconnect to the ONE Platform server to dispose of authentication tokens and create them again. |
13.3.0 |
ONE Web Application |
ONE-24055: It is possible to create new property filters without defining the target Property. This will not function, but validations are not working so no error is thrown. |
13.3.0 |
RDM |
ONE-24105: It is not possible to upgrade Oracle-based projects from version 12.x to 13.0-13.2 due to incompatibility in column naming. |
13.3.0 |
RDM |
ONE-27130: After upgrade, RDM workflows are by default disabled and need to be enabled manually. |
13.6.0 |
MDM |
ONE-24177: Parts of MDM Webapp will not work correctly when app server context path is used. |
13.3.0 |
MDM |
ONE-25239: MDM History Event Handler fails. Workaround: Before migration or upgrade, all events have to be committed and event storage has to be empty. |
13.6.0 |
ONE Web Application |
ONE-24621: Edit button missing for manually uploaded lookup files. |
13.3.0 |
ONE Web Application |
ONE-24745: Unable to add more than 10 catalog items to a monitoring project. |
13.3.0 |
Known Issues Resolved
We report known issues in each release. Below is the list of the issues that have been resolved across the platform for this version.
| Module | Issue | Reported in |
|---|---|---|
ONE Web Application |
ONE-23266: It is not possible to create lookup items from virtual catalog items (VCIs). |
13.0.0, 13.1.0 |
MDM |
ONE-11568: Some features of turning off overrides regarding exports and life cycles are yet to be implemented. |
13.1.0 |
RDM |
ONE-22844: DATE datatype import/export generated as STRING. |
13.1.0 |
RDM |
ONE-22656: DATE datatype cannot be used for validation. |
13.1.0 |
RDM |
ONE-22655: DATE datatype cannot be used for enrichment. |
13.1.0 |
ONE
| If you wish to upgrade from version 13.1.0 to version 13.2.0 of the Ataccama Data Quality and Data Governance suites, contact your account manager to discuss details. |
Recommended Keycloak Version Update
We now support and recommend using Keycloak 13.0.1 version. For a full list of supported dependency versions, see supported-third-party-components.adoc.
ONE Desktop
Support for Salesforce and SAP Servers
We have added support for Salesforce and SAP RFC systems, allowing users to work with Salesforce and SAP data, as well as use preconfigured SAP functions. For more information, see one-desktop:server:connect-to-a-server.adoc, sections Salesforce and SAP RFC respectively.
To be able to consume and modify data from these applications, we have introduced the following reader and writer steps:
-
Salesforce Reader: Reads data from user’s Salesforce application.
-
Salesforce Writer: Allows users to update or delete data in Salesforce SObjects.
-
SAP RFC Reader: Reads data from user’s SAP RFC application.
-
SAP RFC Execute: Allows users to execute SAP functions on SAP RFC data.
ONE Platform Browser Authentication
From ONE 13.2.0, you can use browser authentication for connecting to ONE Platform server.
To connect to ONE Platform using OpenID web browser authentication follow these steps:
-
Select File > New > Server.
-
In the New Server dialog box, select OpenID Connect via Web Browser in Type.
-
Save connection and connect.
-
You will be prompted to allow web browser authentication.
-
Log in using your credentials.
-
After successful authentication you can close the browser tab and return to ONE Desktop.
Support for Nested Components
In 13.1.0 we introduced nested transformation components. A nested component is a transformational component that is used inside another component, which can then be included in complex plans. For example, this lets you cleanse or transform data in virtual catalog items using one of preconfigured components, without modifying the source data. To learn more, see Nested Components.
Your nested components can now be easily imported and converted into ONE components by dragging them from File Explorer into ONE Explorer.
The component file and all linked files will be imported into the componentFiles folder on your ONE Platform Server.
All paths in the component file are converted to relative paths so that they work independently of your local naming conventions.
To import a nested component into ONE Platform follow these steps:
-
Drag a component from File Explorer to any ONE component in ONE Explorer.
-
If there are any external files that the imported component refers to they will be imported too. All paths to external files are modified during import so that the function of the component is intact.
You can also import nested components by dragging them directly onto a ONE component canvas area. To do this:
-
In ONE Explorer, double-click the ONE component that you want to edit in the canvas.
-
Drag and drop any files and components that you want to nest in the component and import to the ONE Platform Server.
-
When you are ready, save your changes, right-click your ONE component and click Publish to apply changes to the server.
Conversion of pathsWhen you drag and drop a file or component onto ONE component canvas the file paths will stay local until you publish the changes to the server. All paths in the component will be converted to relative during publishing of the component.
Configure Components in ONE Desktop
Now you can configure the state and the main file for ONE components in ONE Desktop:
-
Right-click a ONE component and select Configure.
-
You can edit Implementation state and select the main file in order to determine which components are nested.
Enhanced US Address Identifier Step
We have enhanced the function and interface of the US Address Identifier step making sure no data is omitted from the output. Make sure you have the latest lookup files for correct functioning of the Ataccama ONE Platform.
In 13.2.0 we have:
-
Added a new Cleansed tab and added new fields to the Output tab.
-
Reworked the part of validation logic responsible for parsing of different parts of the address, e.g. city, state, units, PO boxes etc.
-
Added a new lookup file to improve unit information parsing.
-
The step supports only 9-digit ZIP codes, so ZIP codes are now trimmed accordingly. 9-digit ZIP codes consist of a State code and ZIP code.
-
Added street addresses to the output for addresses that have been identified as PO BOX type (only the PO BOX part is validated and street address is not).
-
Added new explanation codes related to all recent changes.
ONE Runtime Server
We have added a new Notification Handler Server component, thanks to which users can now schedule running a script or a workflow based on certain changes in Metadata Management Module (MMM). As such, the component is similar to the Scheduler Server Component, with the key difference being how schedules are initiated. The component is integrated in ONE Runtime Server Admin (Notifications tab in Workflows), where you can see the configured notification handlers, keep track of the job execution, as well as unsubscribe the handlers that you no longer need. Information about MMM events can be passed through variables to both script and workflow jobs. For more information about the component, see Notification Handler Component. For more information about how to configure and use notification handlers for different types of jobs, see Notifications Handler.
ONE Web Application
Creating Rules from Analyses
It is now possible to create a rule from Mask, Pattern, and Frequency analysis of an attribute, using the Profile & DQ Insights tab. Rules can be created for the purpose of identifying either correct or incorrect behavior:
-
To create a rule of the correct mask, pattern, or frequency, the
matches maskcondition is introduced. -
To create a rule of the incorrect mask, pattern, or frequency, the
not matches maskcondition is introduced.
Scheduler for Events
In a number of places in the application, it is now possible to set up a schedule for certain actions. These will then be triggered automatically by the application at the specified time.
Currently the scheduler can:
-
Run monitoring projects.
-
Rebuild lookup items.
-
Run the Document, Discover, or Import flow on a data source.
After opening the scheduler from one of these locations, the configuration sidebar opens. This will show existing scheduled events, if any, or you can select Add Scheduled Event.
The timing for the event can be set in two ways:
-
The Basic view of the Scheduler for simpler requirements. Define the schedule via the user interface.
-
The Advanced view of the Scheduler. Define the schedule using Cron format.
Scheduled events can be seen in the Processing Center under Scheduled Jobs.
Managing Top-level Default Access for Entities
With the top-level configuration of any array embedded entity (such as sources, rules, terms), it is possible to set the default permissions, which are propagated to all root entities (to all sources, all rules, all terms). When the default permissions are set on the top-level entities, all changes will be propagated similarly on all available entities in the list.
Viewing Anomalies Tab
Detected anomalies details can be now also found on the Anomaly Overview page which shows aggregated anomalies for all Catalog Items in which they have been detected.
Lookup Improvements
When creating lookup items, you have more advanced configuration options (such as defining supported characters and removing duplicates) and the ability to preview the data of the catalog item from which you intend to build the lookup. You can also now schedule automatic rebuilds of lookup items using the new Scheduler feature. For more information, see Lookup Items.
Build lookups from rules
You can now build lookups automatically during rule definition, using the value is from catalog item feature. For more information, see Use Lookups in Rules.
Ability to Reference Metadata in Rich Text Field
In the rich text fields available in the web application it is now possible to reference metadata, using a forward slash (/) and selecting the required term from the list provided.
Web App Configuration
Available for power users only.
Custom Search Filters
You can now add additional filters to the Catalog Item Search configuration. For more information, see configure-catalog-item-search.adoc.
Detailed Lineage Update
A Manta connection link can be now set to enable direct connection to the Manta graph from the Lineage tab.
The lineage can be also calculated on all available data sources by clicking the Calculate Lineage button on the Sources page.
List of Values Update
It is now possible to assign several values from the list of options. This can be done by creating another entity in Metadata Model serving as a target.
Hide Left Navigation Menu
It is now possible to hide pages from the left-side navigation menu in the case that they should not be seen by the users, but deleting them is undesirable.
Upgrade Center Changes
The Upgrade Center modes were updated and validation checks were added to verify the state of the application before introducing changes. The validation fails can be located and the particular errors identified.
Inconsistencies between roles in Keycloak and their permissions that are identified during the upgrade procedures are now shown on the Root role mapping tab of the Permissions Model page.
Changes in Default Permissions Configuration
In the default permissions configuration, a new capability has been added, governance-editor, which will take the place of the governance-admin capability for the Keycloak role MMM_data-governor.
This means that in the future the data-governor role will not have policy enforcement rights.
For more information, see user-access-management:manage-permissions-users-roles.adoc.
AI Core
In this version, we have made the following changes:
-
There are now two possibilities for upgrading AI Core microservices: manually, by running a Kubernetes job for on-premise deployment, or by restarting the newly added Upgrade microservice (must be enabled first).
-
The Neighbors microservice now uses fixed memory allocation instead of increasing the number of fingerprints as needed. The default maximum number of attributes (1M) can be adjusted for smaller deployments to save RAM. As a result, it is possible to configure a fixed Kubernetes memory limit that is never overstepped. Furthermore, the overall RAM consumption of the Neighbors microservice has been reduced approximately 5 times thanks to a different algorithm applied. The Kubernetes memory limits can therefore be downsized accordingly.
-
Anomaly Detection now supports more generic input built around the category entity.
-
We have added a script for querying the microservice status. The script, located in the build package (ai/bin/healthcheck.sh), produces a human-friendly output containing information about the microservice’s health status and the status of its connection to the dependencies, such as the database and the authentication provider. As the script uses the configuration already provided for the microservice, it can be run directly from the command line without any added arguments.
MDM
Record Labels Update
Record template labels are now generated into mda-label.gen.properties and defined using a more versatile syntax allowing for DQC expressions.
Old definitions can be upgraded using an upgrade template.
See Constructing Record Detail Templates.
Salesforce Stream Consumer
Salesforce can now be used as a stream consumer source. See mdm:input-and-output-interfaces/configuring-stream-consumers.adoc.
Upgrade notes
To determine your upgrade path for version 13.2.0, please see the MDM Upgrade Procedure and MDM 13.2 Upgrade Notes.
RDM
Keycloak Synchronization
There are now several new ways to synchronize roles and users with Keycloak:
-
Automatic sync on user login.
-
Manual sync using a button on the Permissions tab. See setting-permissions-in-rdm.adoc.
-
Scheduled sync using a definition in application.properties. See rdm-application-properties.adoc.
DATE Datatype
The new DATE datatype has been further improved and it is now compatible with the DQC DAY datatype.
Upgrade Notes
There are no version specific upgrade notes for 13.2.0, to determine your upgrade path see RDM Upgrade Procedure.
Fixes
ONE
ONE Desktop
Click here to expand
-
ONE-22549: Server names are not saved in plan metadata anymore. Paths in metadata are now converted to relative paths.
-
ONE-22664: File paths are now converted to relative when you import a component into ONE Platform and convert it to a ONE component.
-
ONE-22731: Components and nested components can now be easily dragged and dropped from File Explorer directly onto your canvas or into ONE Explorer in order to import them.
-
ONE-22533: It is now possible to connect to a server using Keycloak OpenID browser authentication.
-
ONE-23609: You can edit properties of ONE components in ONE Explorer.
-
ONE-23797: We have changed all paths in ONE components to relative.
-
ONE-19142, ONE-21919: We now can identify counties in US Address Identifier step.
-
ONE-23064: Treatment of addresses identified as PO BOX was improved to prevent any issues of data loss.
-
ONE-21090: Records with missing state information are now enriched. 9-digit ZIP codes are now returned for all the addresses that were validated up to BUILDING level.
-
ONE-19031: Support for SAP RFC data source.
-
ONE-20746: Support for Salesforce data source
-
ONE-22837: If there’s a CI attribute of a data type for which there’s no icon (JSON data type) the graph crashes.
-
ONE-23771: The profile results are saved in MinIO forever without any retention.
-
ONE-22362: Profiling table with multiple partitions fails.
-
ONE-23569: When profiling a virtual catalog item terms were applied automatically on attributes as expected, however, parent terms were not tagged on the catalog item as a whole.
-
ONE-23480: It is not possible to add more than 10 rules in the Monitoring Report.
-
ONE-22864: Cannot create the new workflow because of the validations.
-
ONE-23363: Unable to configure mTLS for API calls from plans (for example, using JSON Call step).
ONE Runtime Server
Click here to expand
-
ONE-21390: Event-trigger .EWF workflow runner.
-
ONE-21340: Configuration of logging is not possible via web environment of ONE Runtime Server Admin because the Logging Setup page has been deprecated. Logging can only be configured using configuration files.
ONE Web Application
Click here to expand
-
ONE-22368: Direct link to Manta can be configured from the Lineage page.
-
ONE-21399: Multiple values can be assigned from the list.
-
ONE-22367: Relation Graph shows the custom entities.
-
ONE-23333: It is now possible to schedule Documentation Flow runs from the Sources connection.
-
ONE-22782: The Anomalies Overview page was added, which aggregates all catalog items with anomalies in one place.
-
ONE-23683: The link renderers are configurable and validated from Debagging Tools.
-
ONE-23526, ONE-23946: Upgrade Center Validations are introduced to verify the application status and search for issues and incompatibilities before the upgrade.
-
ONE-23549: With the top-level permissions setting on the nodes it is now possible to edit the default permissions on the high-level nodes (terms, sources, rules).
-
ONE-22889: It is now possible to schedule the runs of the monitoring projects.
-
ONE-22890: Scheduled events can be viewed in Processing Center.
-
ONE-23158: It is now possible to schedule events on terms.
-
ONE-21622: It is possible to create rules directly from mask, frequency, and pattern analysis in Catalog Items profiling results.
-
ONE-21582: Catalog Event API.
-
ONE-22964: Rules with dateAdd function fail in Spark jobs.
-
ONE-23654: Invalid samples sometimes display float numbers in scientific notation.
AI Core
Click here to expand
-
ONE-22615: Upgrade microservice for running DB upgrades.
-
ONE-23329: Limit for neighbors fingerprint index size.
-
ONE-19661: Pre-allocate memory for fingerprints in neighbors on start.
-
ONE-19075: Generic API for Anomaly detection implemented, new structure of categories, optimization of code.
Data Processing Module and Data Processing Engine
Click here to expand
-
ONE-21484, ONE-16106, ONE-17097: Reworked engine registry with more resilient status checking and new configuration parameters.
-
ONE-22552: Improved the efficiency of the internal job queue.
-
ONE-21548: Run results not being persisted.
-
ONE-22376: DPE (cloud deployment) does not start after graceful shutdown.
ONE Spark DPE
Click here to expand
-
ONE-22183: ONE Spark DPE is now able to use Azure Managed Identity Services to authenticate using Azure KeyVault directly.
-
ONE-22324: It is now possible to connect to your Databricks cluster using Azure Active Directory access token.
-
ONE-20787: Issues with correct job routing.
-
ONE-23012: We have added a new configuration parameter that determines automatic mounting of Databricks. Set #plugin.executor-launch-model.ataccama.one.launch-type-properties.SPARK.dbr.autoMount to false to disable automatic mounting.
MDM
Click here to expand
-
ONE-24093: Editing record details with lookups throws error.
-
ONE-23985: Comments not stored in database.
-
ONE-23057: Output columns on read step.
-
ONE-22801: Salesforce stream consumer support.
-
ONE-22779: Case sensitive parameter.
-
ONE-22735: UI translation into Russian.
-
ONE-21981: Override icon not displayed on detail screen.
-
ONE-21933: Issues with high number values.
-
ONE-21761: Escaping of the # character in issue description.
-
ONE-12301: New label definition.
-
ONE-12086: Rematch behavior on master detail view.
Was this page useful?