Ataccama 17.2.0 Release Notes

In the Test rule dialog, DateTime values are displayed in the same time zone in both the test values and the rule conditions.

On the catalog item Data Quality tab, rules with a configuration problem, such as an unmapped input or an input of the wrong data type, are now marked with a warning icon that explains the problem.

When creating a rule, the Stewardship field shows only the groups you’re a member of, and their parent and child groups. If you have the MMM_admin identity provider role, you can select any group. Existing rules keep their current owner group. See Stewardship groups for new rules.

Catalog item attributes whose data type was stored in lowercase or mixed case, such as string instead of STRING, no longer cause errors or get skipped when generating ONE expressions with AI, applying DQ rules to attributes, deploying DQ firewalls, profiling, or exporting data. See Attribute data types are stored in uppercase.

You can no longer delete a lookup that is used in a published rule. Previously, deleting it broke the rule and resulted in failed DQ jobs. See Delete lookups.

After upgrading to 17.1.0, monitoring projects whose post-processing plan includes a Database Output step no longer fail validation with There is no attribute named 'not_applicable_rules' in the catalog item. The not-applicable result columns are now added only when the project has a rule that can produce a not-applicable result, so existing plans validate and run without changes.

Renaming a result in a DQ dimension applies to monitoring projects even without republishing the affected rules. Previously, after such a rename, monitoring projects could fail during pushdown processing or produce incorrect DQ results. Deployed DQ firewalls affected by the change are redeployed automatically.

When importing a monitoring project configuration into another monitoring project, post-processing plans are now copied even when a catalog item is mapped to a different one than in the source project.

Monitoring project transformation or remediation plans with a ONE Data Reader step no longer fail with an Eager fetching violated error at runtime.

In historical runs of a monitoring project, the second and later pages of the Items to monitor list on the Configuration & Results tab load correctly instead of appearing empty.

On the monitoring project Overview tab, the Restore rules action is now deactivated when there are no attributes to restore.

Improved logging of Microsoft Teams notification delivery to help troubleshoot failed webhook calls.

On the catalog item Data Quality tab, rules that aren’t supported in pushdown and are therefore skipped during DQ evaluation are now shown as deactivated, matching the existing behavior in monitoring projects.

Pushdown compatibility warnings on rules, rule variables, and terms are displayed to all users with access to the item, not only to admins.

The Add data remediation plan option is no longer offered on catalog items with pushdown enabled, since data remediation plans cannot run on pushdown.

The Legacy pushdown option is no longer displayed as enabled on the connections detail page when pushdown processing is disabled for the connection.

Pushdown processing can now profile tables with over two billion records.

Snowflake profiling and DQ statements that run longer than a few minutes no longer fail with a driver NullPointerException after repeated re-execution.

DQ pushdown on BigQuery with invalid-record sampling no longer fails with an out-of-memory error on tables with millions of rows.

Monitoring projects with several large lookups on Databricks pushdown no longer fail with ConcurrentAppendException during lookup maintenance.

is null conditions on complex columns, such as MAP, no longer fail on Databricks pushdown.

concatenateif with an empty separator works in pushdown. Previously, BigQuery rejected the generated STRING_AGG with a NULL argument.

BigQuery profiling no longer logs Access forbidden to file …​/protobuf-java-*.jar errors in secured environments.

The attribute selection in the Join and Delete attributes transformation plan steps now consistently displays all newly added input attributes. Previously selected attributes that no longer exist on the input are marked with a warning so you can remove them.

Data slices with a custom date range ending on the maximum date, 9999-12-31, work as expected, and records with this date are included in the slice. Previously, depending on the database and column data type, jobs using such a slice failed with a date conversion error, or returned no records when the dates were stored as text or numbers.

A data slice with a custom date range can no longer be saved when the From date is later than the To date, and the tooltip explains why. Previously, such a slice could be saved but selected no records.

The tooltip on a deactivated Create data slice action now states the actual reason, such as a missing connection or an unavailable connection driver, instead of always saying that the connection doesn’t support data slices.

When a DQ firewall fails to deploy, all users the firewall is shared with receive an in-app notification. Previously, the failure was shown only on the firewall’s detail page.

DQ firewalls handle heavy load more reliably, with a lower risk of running out of memory and responding with 504 Gateway Timeout errors. They now size their JVM heap at 70% of the available memory instead of using a fixed default.

In reconciliation project results, the Compare view displays the profiles of the compared attributes side by side.

When selecting sources in reconciliation projects, the structure of the catalog item source loads reliably. Previously, it could fail to load, especially for sources with many nested levels or after switching between sources, and the selected item had to be removed and added again.

The profile inspector opened from a monitoring project shows all attributes with anomaly detection enabled, including those for which anomaly detection has not produced results yet.

Anomaly Detector installed from the Linux package no longer crashes repeatedly on startup on RHEL 9 and other Enterprise Linux 9 systems, for example after an operating system update. Deployments using the container image were not affected.

When importing metadata for a Microsoft Excel file, you can now configure how many rows are used to detect the column data type (by default: the first 50 rows of the sheet). See How column data types are detected.

Full profiling and Full profile & DQ evaluation runs on a catalog item now form a single profiling history. Previously, each run type was tracked separately, so the history graph displayed only one type of run and anomaly detection for one type could not use the results of the other.

Removing an attribute from a catalog item, for example after its column was deleted in the data source, no longer discards that attribute’s results from previous profiling runs. Profiling history keeps showing the attribute as it was at the time of each run.

Failed profiling jobs show the actual cause of the failure instead of only the generic Unexpected error occurred, job failed in the transaction roll-back. message.

Exporting profiling and DQ insights to a Microsoft Excel file no longer fails when a catalog item has attributes whose names differ only in letter case.

Profiling no longer fails for columns whose name contains ].

Metadata import from Snowflake no longer fails for the entire schema with a No enum constant net.snowflake.client.jdbc.SnowflakeType error when a table contains a column type the driver doesn’t recognize, such as ROWINDEX. Such columns are instead skipped and a warning is logged.

Salesforce connections using OAuth no longer fail to log in with an Unrecognized field "token_format" error after the Salesforce External Client App is reconfigured or recreated.

The Salesforce connector’s REST API version is now configurable using the -Dsalesforce.apiVersion JVM system property. The default remains 50.0.

BigQuery connections using OAuth user credentials no longer fail to connect when the client secret is stored in an external vault (Azure Key Vault, HashiCorp Vault, AWS, or CyberArk).

Browse and preview results are no longer cached for data sources configured with user impersonation, so results always reflect the requesting user’s own credentials.

Table and column previews while browsing a database connection use the interactive credentials, as the preview of imported catalog items already did, instead of requiring default credentials.

Browsing a database no longer retrieves primary and foreign key metadata that isn’t needed to list table and column names, so browsing works for users without permissions on that metadata.

Fixed out-of-memory errors when browsing schemas with a very large number of tables. Table details are now loaded only for the page being viewed, and very large browse results are temporarily stored on disk instead of in memory.

When browsing a file system source, import settings set on a folder are applied to the catalog items imported from that folder, so attribute names and data previews are correct.

Saving a folder import setting in Connection Import Settings no longer fails to publish with a FOLDER_SETTING_FORMATS constraint violation when the setting had a leftover draft.

The credentials detail shows the credential type, so you can tell which of several similar types, for example for Amazon S3, a connection uses.

In the SAP OData connection settings, the OAuth credentials section shows its correct label.

Catalog item preview now works for Tableau reports hosted on the default site, where the site content URL is empty.

Long error messages from DPE, such as those from certain ADLS connection test failures, are no longer replaced with a generic RST_STREAM closed stream. HTTP/2 error code: PROTOCOL_ERROR error. These messages are now truncated so the actual cause is shown.

Metadata import, data observability, and profiling now read from the correct source when jobs run at the same time on two or more connections of the same database type, for example, two PostgreSQL connections. Previously, this could return tables, schemas, or results belonging to a different source.

Long-running JDBC operations no longer fail intermittently with a NoClassDefFoundError, typically on Azure MS SQL connections using service principal authentication.

In Ataccama Cloud environments, DPE no longer runs out of memory and becomes unavailable with a There is no available data processing engine. error after browsing several different sources.

In Ataccama Cloud environments, job engines are sized correctly for all environment sizes, so memory-heavy jobs such as profiling Parquet files are no longer terminated without notification.

Jobs stuck in DPE, either accepted but never started or canceled but never confirmed, are now automatically failed after a configurable timeout instead of blocking the queue indefinitely.

A single job event that can’t be saved no longer blocks event synchronization for that DPE. Previously, an event containing a NUL character (for example, from a data source driver’s error message) could prevent all later events, including job completions, from synchronizing until DPE was restarted.

Jobs no longer get stuck in the RUNNING or KILLING state when they lose their engine assignment. Previously, the status update failed and was retried forever, and the job held its allocation until the Metadata Management Module was restarted.

In Ataccama Cloud environments, when the container of an autoscaled job engine is terminated, for example because it ran out of memory, the job fails right away with the actual cause instead of after an hour with The timeout for reconnection of the engine has expired.

Finished jobs are correctly removed from the Data Processing Module (DPM) database once they expire. Previously, in 17.1.0, the cleanup failed, so the database kept growing and the log filled with cleanExpiredJob status=FAILURE warnings.

DPM recovers from a brief unavailability of its database without a restart.

DPE recovers its connection to DPM after a connection failure under heavy job load, instead of staying disconnected and making DPM report as not ready until a restart.

Profiling no longer starts with a delay of several minutes after a metadata import in environments with millions of catalog items. The import status in ONE now matches the DPM Admin Console.

Job failure reasons are now limited to 8,000 characters, keeping the beginning and end of the message. This prevents failing jobs from bloating database storage and slowing down full-text job search, and the full output remains available in the job log.

Structure check on spreadsheet sources from local file systems now always reflects the file’s current schema. Previously, results could be incorrect for up to a few minutes after the file was browsed or profiled.

Monitoring projects on catalog items from a local file system with Spark post-processing plans no longer incorrectly fail model validation.

Monitoring projects that combine a Spark Writer post-processing plan with another transformation plan no longer fail with Step cannot run locally [Spark Writer]. Previously, the job was incorrectly run locally instead of on Spark.

DQ evaluation on a source processed on Spark no longer incorrectly runs on zero records while reporting success. Previously, such jobs could finish successfully with all DQ results empty (N/A) and write an empty target table.

Spark processing no longer fails in environments that use unique encryption keys for secrets.

Concurrent Databricks jobs on a shared cluster no longer fail with a Plan file does not exist error.

Multipart uploads to Amazon S3 no longer fail with The request content has fewer bytes than the specified content-length when the AWS SDK retries a part.

Logging out of the DPM Admin Console no longer fails with an invalid redirect URI error.

In self-managed deployments, the Keycloak start.sh script sources an optional conf/start.sh.local file, so customizations such as reading the database password from AWS Secrets Manager survive upgrades.

Hybrid DPE now connects with encryption enabled in Ataccama Cloud environments. Previously, required encryption properties were missing from its configuration, so the connection failed.

The generated values.yml for Kubernetes hybrid DPE deployments includes the image pull secret under the correct name.

Encrypted values (crypted:AES:…​) are accepted when properties are passed as environment variables, not only in application.properties.

In Ansible deployments, the DPE log file rollover size is configurable instead of fixed at 10 MB.

Saving records in a ONE Data table no longer fails with an error when a rule assigned to the table is outdated, for example after you edit a rule that is already assigned. The outdated rule is marked with a warning and skipped until you update it, and the other rules on the table continue to be evaluated.

Repeated data remediation runs no longer affect user access to data remediation output tables in ONE Data.

Date values entered or pasted into ONE Data are now interpreted consistently across supported date formats.

Updated permission checks for data protection classification.

On the term Settings tab, detection rules removed from the term are no longer shown as selected in the Add Rules dropdown and can be assigned again. Assigned rules also no longer lose their names and types in the Detection Rules section after you select Request review and publish. Previously, both issues needed a page refresh to clear.

Accepting a term suggestion without publishing it no longer makes saving and publishing the asset fail with the REFERENCED_OBJECT_DOES_NOT_EXIST error after the suggestion is deleted, for example, when term suggestions are refreshed. If this still happens after the upgrade, contact Ataccama Support.

Search no longer returns empty results after an upgrade when the search configuration references properties that no longer exist in the model. Such references are now skipped and reported, and all remaining content is indexed as usual.

When you delete a custom property that is indexed in a search configuration, the stale reference is now removed automatically when the model change is applied, so it no longer causes repeated Unknown property errors in the logs. Existing stale references are cleaned up during the upgrade to 17.2.0.

Full-text search now returns correct results when searching for assets whose names contain underscores (_).

The Terms screen now loads when its search configuration indexes a computed content property as a child node. Previously, publishing this configuration could prevent the screen from loading and showed a Renderer for column "name" not found error instead.

When listing the terms by relationship on screens under Terms, such as Business Terms, the Published, Unpublished, and All tabs are shown only when full-text search is enabled. Previously, selecting one of these tabs locked the listing results until the page was reloaded.

Stewardship changes now follow the Review workflow configured for the asset type, like other edits of an asset: you publish them yourself, or, if the asset requires review, they create a review request. For asset types that are published directly, without drafts, such as sources and connections, stewardship changes are still published immediately. See Stewardship changes follow the Review workflow.

Improved authorization for review requests and data access requests: only authorized approvers can approve or reject a request, and requesters can no longer close or delete pending approval tasks. Publishing an asset governed by an approval workflow directly via API requires an approved review request. See Review and Request Data Access workflows apply to ONE API clients.

The Request review and publish bulk action is also available on the Catalog Items > Unpublished page, so you can request a review for multiple catalog items in a single review request. The action was already available for terms, monitoring projects, and rules.

Submitting a review request for a catalog item with many related assets no longer fails with the Exceeded limit on max bytes to buffer : 5242880 error. See The Workflow service memory limit uses a new property.

In monitoring projects, a review request could become stuck with no way to approve or reject it when a second review request was created against the same underlying task. Such invalid review requests are now rejected at creation.

Task descriptions created via API, for example by a JSON Call step in a post-processing plan, keep their line breaks when displayed in the task detail. Previously, line breaks were visible only when editing the task.

The I’m assignee filter on the Tasks Overview screen shows tasks assigned to a group you’re a member of, not only tasks assigned to you directly, consistent with the Tasks widget on the home page.

Imported assets that matched an existing draft with identical content in the target environment are now included in the import and published together with it. Previously, they were left out, so publishing the import either kept the old published version of the asset without any warning or failed with Parent entity must be published first.

Deleting an import job that hasn’t been published now also discards the drafts the import created. Previously, the drafts stayed in the target environment with no way to publish, discard, or inspect them, and they affected how later imports of the same assets were matched.

Exported archives now include assets that are referenced only indirectly, through other referenced assets. Previously, the export finished without a warning, and the import reported unresolved references if the missing assets didn’t exist in the target environment.

Run validations only in the Drafts data state now applies the same ID mappings as the actual import, so its report matches what the import would do. Previously, the validation matched assets by name only and could report assets as new or references as unresolved when the import would have matched them by ID.

A failed import no longer ends with an empty validation report and an unrelated error about /jobs instead of the actual validation issues. When an import is rolled back, the job now states Import was rolled back, nothing was imported.

The downloadable import report now includes all errors, not just the first page.

Unexpected errors during import are easier to diagnose: their details are recorded in the validation report and in the application log.

Importing an asset that matches an existing asset of a different type by ID or keys no longer fails. Assets are matched only to assets of the same type.

Importing a rule whose condition uses a lookup built from a catalog item no longer fails.

Validating a rule that uses a lookup existing only as a draft, for example when exporting a monitoring project, now reports that the lookup is unpublished and must be published first, instead of Reference data lookup with id or name used in the rule not found.

Catalog item pages no longer fail with Error occurred during layout render for manually created file catalog items.

Users with view-only permissions on a catalog item can no longer open edit mode. Previously, edit mode was accessible even without edit rights, although changes could not be saved.

Custom previews on report catalog items no longer disappear after you refresh the page. Previously, the preview could stop loading once its cached download link expired.

On attribute pages with a customized layout that includes the Relationships widget, the relationship picker shows the full path of each candidate attribute instead of just a name, so attributes with identical names can be distinguished.

The Ataccama ONE version under My Account > Version Information shows the exact patch version, for example, 17.2.0-patch1 instead of 17.2.0.

Reduced load on the MMM database when indexing catalog items for search in environments with many catalog items, for example after importing or profiling sources that change many catalog items at once. As a result, new and updated catalog items now appear in search results faster.

Term suggestion processing no longer retries updates it can never process. Such updates are logged once and set aside, so they no longer flood the logs with errors or keep the MMM database under sustained load.

Reduced database load from term suggestion queries in large environments.

Reduced log noise from expected permission denials, such as the pushdown availability checks that run automatically when rules are listed. These OPERATION_NOT_ALLOWED events are now logged as warnings. Denials from actions a user explicitly attempted are still logged as errors.

Data quality results over a time range can be retrieved through the GraphQL API with only a start date or only an end date specified. Previously, such requests failed with an internal error.

When retrieving data quality results through the DQ&C MCP server, the overall data quality result is returned separately and no longer appears as an Unnamed dimension.

The overall data quality result displayed in Atlan is now rounded the same way as in ONE, so both show the same value.

The DQ&C MCP server tools that return catalog item details and attribute profiling now use the latest profile of the catalog item and state when it was taken. Previously, they returned the first profile, even after the catalog item was profiled again.

AI clients connected to the DQ&C MCP server stay signed in when their access token expires. Previously, you had to sign in again every time the token expired.

Security improvements in DQ firewalls. See also gRPC reflection turned off by default on DQ firewalls.

After signing in, you are always returned to a page within the application.

Upgraded third-party dependencies in DPE, DPM, and the DPE JDBC drivers for enhanced security.

Matching no longer merges records that don’t match under the same master record when a large key group is processed in parallel. Previously, repeated runs of the same data produced a different, non-deterministic number of master records.

Matching with constraint rules no longer runs out of memory on large matched groups. Match pairs that don’t fit in memory are written to temporary files, so very large groups use additional temporary disk space instead.

Master records that lose their authored nature are no longer offered as match candidates, and their rows in the authored-master key table are removed when the nature changes. Rows orphaned before this fix are not cleaned up automatically.

MDM master records no longer lose their column override values after a parallel-write conflict on the same master record. Previously, the overridden value could be replaced by the newly consolidated value while the override itself appeared unchanged.

The Prepare subtask in merge actions is now faster. Previously, it applied an unnecessary override-table join that could severely slow down loads after upgrading from 13.x to 15.x or later.

The streaming event handler now publishes instance records of second-layer entities (for example, address instances under an address master) even when the intermediate master entity has no changes of its own.

The streaming event handler no longer fails publishing with a NullPointerException when a child entity in a traversing plan has an empty reference to the root entity (for example, a shared entity linked to a different parent). Such records are published only through the traversing plan that reaches them via the parent they are linked to.

Activating a streaming event handler in the MDM Web App Admin Center now retries publishing sessions that crashed, the same way the asynchronous event handler does. Previously, the handler deactivated itself again and the only way to recover was to remove the crashed tasks, which discarded their events.

MDM streaming event handlers no longer fail to persist large JSON payloads on Oracle and SQL Server databases for new tables. Previously, payloads larger than 4000 bytes failed with an ORA-12899 error on Oracle. Existing tables require a manual DDL migration.

The history event handler no longer fails with a NullPointerException when metadata such as the modifying user is missing, which previously blocked downstream history events.

In ONE Desktop, the generated streaming event handler configuration no longer combines the names of two relationships when a related entity has more than one relationship to the publisher root. Previously, MDM Server failed to start with Relationship '…​' not found.

The MDM Web App Admin Center now shows the full deployed version of MDM Server, including the patch number, so you can tell at a glance which patch your environment runs.

Event handler task details now open while the task is publishing, instead of failing with Unable to retrieve the event handler task details.

The batch load and export Run actions are grayed out on read-only and high-availability (HA) passive nodes. Previously, the actions stayed available and the operation failed only after it was started.

The Download operation plan in XML link on the Operation Plan page downloads the plan again, instead of redirecting to the Load Operations page.

In the stop server dialog, the View all link for running workflows opens the list of running workflows, instead of the Load Operations page.

Users without an MDM role can no longer call the REST API, and listing or changing event handlers and terminating running tasks over the REST API now require the operational permissions, consistent with the Admin Center. See REST API security.

Creating tasks no longer fails when Keycloak groups have a description set, which previously left the assignee list empty.

MDM Web App now works correctly when deployed under a servlet context path (server.servlet.context-path, or global.mdm.contextPath in cloud deployments). Login, MDM Web App Admin Center, links, and images resolve under the configured path. See MDM Web App Application Properties.

In the legacy interface:

Workflow transitions in GUI Configuration no longer fail validation with You cannot use disabled step for newly created workflow steps. The per-step Enable option was removed in an earlier version, but the validation still checked it.

Data navigation folders whose names contain digits, such as Bank 1, are now generated.

Content generators normalize entity, layer, and relationship names consistently, so labels, enrichments, autocorrects, custom actions, data samples, and search filters are generated for entities with mixed-case names. Technical names used in file names and identifiers, such as workflow, custom action, and dataset names, are now validated.

MDM Server no longer fails to start with batch export plan …​ contains input 'eh_subtasks' which does not map to the provided data when Enable Event Handler Status is selected in Task Information Export.

In the MDM example project, the full_reprocess_operation_rematch operation now rematches with the Remove manual matches strategy, and the generated technical documentation shows the correct Rematch Enabled value for reprocess operations. Previously, the operation ran a plain reprocess.

On Windows, the Keycloak demo script start-demo.bat creates the ataccamaone realm again.

In cloud deployments, the MDM Server can authenticate to AWS services with AWS_WEB_IDENTITY_TOKEN or EKS Pod Identity instead of static access keys, so deployments on Amazon EKS no longer need to store and rotate IAM credentials.

System indexes are now built in parallel, which shortens initial loads that drop and rebuild the indexes.

Data statistics no longer drift during highly parallel commits (commitParallel greater than 1), which caused active and inactive record counts to be incorrect.

Instance records no longer end up with an empty eng_source_timestamp after a parallel-write conflict resolution.

The read-only mode of an MDM Server node is set with the new ataccama.one.mdm.nme.readonly property, so it can again be set per node instead of being shared through the nme.readonly runtime parameter. This lets you run a dedicated read-only node in a multi-node deployment. See MDM Server Application Properties.

Upgraded third-party dependencies in MDM Server and MDM Web App for enhanced security.