User Community Service Desk Downloads
If you can't find the product or version you're looking for, visit support.ataccama.com/downloads

Ataccama 13.5.0 Release Notes

This page contains the release notes for version 13.5.0 of the entire Ataccama ONE Gen2 platform. Release information affecting all modules in the platform can be found directly below; module specific information can be found in the ONE, MDM, and RDM sections.

Release Information

Products

ONE Data Governance, ONE Data Quality and Governance, ONE RDM, ONE MDM

Release date

2022-01-06

Downloads

support.ataccama.com/downloads/13.5.0

ONE

Secure Communication Between DPE and DPM

All DPE instances need to be registered individually with DPM: during this registration they can announce whether or not they are TLS enabled. TLS is required to ensure secure communication between DPE and DPM in hybrid deployments when firewall-friendly communication cannot be used. Previously, all connections had to use either secured connections or none could.

ONE Desktop

New Driver Parameters Field for Amazon Redshift

We have introduced a new field called Driver Parameters in the Database Connection Details window to provide additional configuration for Amazon Redshift connections. By default this field will contain the parameter fetchRingBufferSize=1M. This is optional but recommended as it modifies the default buffer size of the Redshift driver (2GB) to fit the maximum memory capacity of ONE Desktop (1 GB), so prevents memory-related issues when fetching a large number of records.

ONE Web Application

Change in DQ Storage

This version includes changes in DQ storage which greatly improve performance when saving and querying DQ results. This change affects all APIs querying monitoring project results however, which in turn means that ONE metadata reader step can no longer be used to access DQ results, and that from version 13.5.0 results will be accessible from plans only via API.

User Upgrade Commands

In addition to system upgrades, the Upgrade Center is now also used for applying model changes. This means that, starting from this version, changes made to the metametadata (MMD) model are published using the Upgrade Center instead of an approval workflow. When users create a draft version of an entity in the model, the Upgrade Center generates an appropriate upgrade command that is listed in the Upgrade Center under Pending Changes. Users can also upload custom upgrade commands.

ataccama 13.5.0 release notes upgrade commands

As a preparation step, the upgrade procedure includes creating a database backup before applying changes. In addition, changes are validated before they are published and users are notified of any conflicts. For more information, see System Changes

Audit UI

Audit logs can now be visualized in the web application. Provided that the Audit Module is configured for your ONE instance, go to Organization > Audit to access the audit information. Here you can see a list of all collected audit logs as well as display related operations and affected assets when viewing an entry’s details. For more information, see Audit.

ataccama 13.5.0 release notes audit

Exporting Audit Logs

Depending on your distribution, the Audit Module may include the Export feature. When the Export feature is configured, audit logs are exported in CSV format to a designated ONE Object Storage bucket following a predefined schedule. For more information, see Audit, section Exporting Audit Logs.

Spark Processing of Relational Databases

In order to process large volumes of data you can use your Spark cluster. This will improve how ONE works with relational databases when profiling them and during data quality tasks. Your Spark cluster can connect to an RDBMS database (e.g. Oracle, SQL Server) with multiple JDBC connections at once in order to retrieve the data or use open sourced Spark Connectors for cloud DWHs (e.g. Snowflake, Azure Synapse). In the latter case, the data for processing is quickly moved from the DWH to Databricks cluster using ONE Object Storage.

Filters in DQ Reports

It is now possible to use attribute filters in DQ Reports. If a catalog item has attributes for which the 'Filter by' option has been enabled in the project configuration, the same filters will be available in the DQ Report tab, on the catalog item level.

Configurable retention period for monitoring project results

Both at a global level (power-users only) and at a project level, you can configure how long you would like to store monitoring project results. If both global and project settings are defined, global settings take precedence.

Aggregation Rules

Previously, in order to use aggregate functions in DQ rules it was necessary to use ONE Desktop. Now, aggregate functions are available in the rule logic for DQ rules. You can also use the new Group by section to separate data into multiple groups of rows based on the expressions provided. The primary use case for aggregation rules—uniqueness—has been provided as a business friendly rule, meaning it can be configured even using just basic grouping by attribute (or attributes) and the Condition Builder.

This means there are three possible methods of rule creation:

  1. Use of aggregate functions in Rule Logic Advanced Expressions.

  2. Grouping of the data (using Group by) followed by use of aggregate functions in Rule Logic Advanced Expressions.

  3. Grouping of the data (using Group by) followed by use of hidden aggregate function in the Rule Logic Condition Builder.

MDM

In this version, native services available on HttpEndpoint in both SOAP and XML formats are not secured by default and additional configuration must be used. For more information, see Native Services.

Actuator and Prometheus Support

MDM Server now exposes standard production-ready endpoints for application monitoring and management:

  • Prometheus endpoint

  • Actuator endpoints, e.g. /info and /health

All endpoints can be secured by selected authentication mechanism:

  • basic authentication — the recommended default

  • JWT (Keycloak)

  • internal security

Default settings (defined in MDM Server Application Properties):

  • authentication role ONE_PLATFORM_MONITORING (required if security for these endpoints is turned on, name can be configured)

  • ports:

    • MDM:8051 → MDM:8057

    • MDM webapp:8050 → MDM webapp:8056

  • /info and /health endpoints are not public

  • on-premise = app as is (secured, dedicated port)

  • cloud = docker (unsecured, dedicated port)

application.properties endpoints example
# endpoints for monitoring
management.endpoints.enabled-by-default=false
management.endpoint.health.enabled=true
management.endpoint.health.show-details=always
management.endpoint.health.show-components=always
management.endpoint.info.enabled=true
management.endpoint.prometheus.enabled=true
management.endpoints.web.exposure.include=health,info,prometheus
management.endpoint.health.status.order=down,out-of-service,unknown,up
management.info.git.mode=full
# Timing metrics to all spring endpoints (true by default)
management.metrics.web.server.request.autotime.enabled=false

VLDB Commands Improvement

VLDB persistence is able to run custom SQL during inlist after keys are inserted into INL_S or INL_L table and before running JOIN of inlist and data table. Symbolic names {inlist}, {data} and {keyName} can be used. See database-hints.adoc.

Custom VLDB command example
<vldb-commands>
    <inlistBeforeCommand>
        <variants>
            <commandVariant>
                <expectedDataSize greaterThan="15" />
                <template>
                    INSERT INTO mdc_table_assert(id, val) VALUES(1, '{data}-{keyName}-{inlist}')
                </template>
            </commandVariant>
        </variants>
    </inlistBeforeCommand>
</vldb-commands>

Upgrade Notes

There are no version specific upgrade notes for 13.5.0, to determine your upgrade path see MDM Upgrade Procedure.

RDM

Upgrade Notes

There are no version specific upgrade notes for 13.5.0, to determine your upgrade path see RDM Upgrade Procedure.

Was this page useful?