ONE Desktop ONE DQ&C ONE MDM ONE RDM ONE Runtime Server
latest (15.4.0) 15.3.0 15.2.0 15.1.0 14.5.x 13.9.x 13.8.x 13.7.x 13.6.x 13.5.0 13.4.x 13.3.x 13.2.x
User Community Service Desk Downloads
If you can't find the product or version you're looking for, visit support.ataccama.com/downloads

MDM 15.4.0 Upgrade Notes

Native Amazon SQS streaming connector

We have introduced native support for Amazon SQS to replace the previous JMS-based implementation.

To transition to the new integration, after the upgrade, follow one of the two approaches:

  • Create a new SQS connector and migrate the logic:

    1. Create a new SQS connector: Set up and configure the new native SQS connector with the required parameters and settings.

    2. Migrate the logic: Transfer the configuration and processing logic from the old JMS stream source to the new SQS connector.

  • Rename the old JMS stream source and create the new SQS connector:

    1. Rename and disable the old JMS stream source: Change the name of the existing JMS stream source to disable it.

    2. Create the new SQS connector: Set up the new SQS connector using the original name of the JMS source to ensure that existing plans automatically link to the new connector.

The JMS-based SQS connection is deprecated but will continue to be supported for a transitional period.

New Prometheus metrics for Amazon SQS

We introduced new Prometheus metrics to monitor the integration with Amazon SQS, that provide details about message flow, processing rates, and potential issues.

Metric Description

mdm_stream_sqs_input_queue_duplicated_total

Total number of messages removed from the input queue due to duplication.

mdm_stream_sqs_input_queue_limit_total

Maximum allowed input queue length.

mdm_stream_sqs_input_queue_pop_total

Total number of messages removed from the input queue.

mdm_stream_sqs_input_queue_push_total

Total number of messages added to the input queue.

mdm_stream_sqs_queue_delete_batch_count_total

Total number of non-empty delete batches sent to the SQS client.

mdm_stream_sqs_queue_delete_batch_limit_total

Maximum allowed delete batch size.

mdm_stream_sqs_queue_delete_failed_total

Number of failures encountered when attempting to delete messages from SQS.

mdm_stream_sqs_queue_delete_message_count_total

Total number of messages deleted from the SQS client.

mdm_stream_sqs_queue_receive_batch_count_total

Total number of non-empty batches received from the SQS client.

mdm_stream_sqs_queue_receive_batch_limit_total

Maximum allowed receive batch size.

mdm_stream_sqs_queue_receive_failed_total

Number of failures encountered when attempting to receive messages from SQS.

mdm_stream_sqs_queue_receive_message_count_total

Total number of messages successfully received from SQS.

mdm_stream_sqs_state

Current state of the SQS connection.

History plugin configuration

The history plugin is now configured in the application.properties file with these default values (see MDM Server Application Properties, section History plugin configuration):

ataccama.one.mdm.config.historyConfigFile=nme-history.gen.xml
ataccama.one.mdm.nme.history.eventPersistence.type=FILE
ataccama.one.mdm.nme.history.persistence.class=com.ataccama.nme.persistence.vldb.VldbPersistenceFactory
ataccama.one.mdm.nme.history.storage-directory=../storage/history
ataccama.one.mdm.nme.history.persistence.dataSource=mdc_db
ataccama.one.mdm.nme.history.persistence.prefix=H_

After the upgrade, move the history plugin configuration from the nme-config.xml file to the application.properties file. We recommend using the default values for the history plugin configuration.

Matching step parallelism configuration

We introduced a new configuration approach for matching step parallelism. It allows you to specify the number of threads and data thresholds at the partition, key, and rule levels. To learn more, see Runtime Parameters.

The previous parallelism settings are now deprecated but will be supported for a transitional period.

Migration from DGS to Spring GraphQL

dgs.graphql.* properties were renamed to spring.graphql.*, see DGS documentation. Specifically, dgs.graphql.introspection.enabled was renamed to spring.graphql.schema.introspection.enabled.

After the upgrade, follow these steps, otherwise, MDM Web App will fail to start:

  1. Remove the properties ataccama.one.mda.server.graphql.* from the application.properties file.

  2. Rename the dgs.graphql.introspection.enabled property to spring.graphql.schema.introspection.enabled.

  3. Change the property names dgs.graphql.* to spring.graphql.*. Alternatively, remove those properties to use the default values (specifically, spring.graphql.schema.introspection.enabled=false).

Configuration changelog

This section provides the information about the changes in the configuration files that need to be taken into account when upgrading the solution. In certain cases, the newly introduced default is used or some configuration is required.

The following table lists the configuration files that were added, changed, or removed.

Added files Changed files Removed files

/

  • MDM Server application.properties

  • nme-config.xml

  • runtime.parameters

/

The following sections list the changes in the files: added, changed, and removed properties.

MDM Server application.properties

Added properties

  • History plugin configuration: move the configuration from the nme-config.xml file to the application.properties file, see History plugin configuration.

    ataccama.one.mdm.config.historyConfigFile=nme-history.gen.xml
ataccama.one.mdm.nme.history.eventPersistence.type
ataccama.one.mdm.nme.history.persistence.class=com.ataccama.nme.persistence.vldb.VldbPersistenceFactory
ataccama.one.mdm.nme.history.storage-directory=../storage/history
ataccama.one.mdm.nme.history.persistence.dataSource=mdc_db
ataccama.one.mdm.nme.history.persistence.prefix=H_

  • ataccama.one.mdm.nme.history.eventPersistence.type - the default value FILE, see MDM Server Application Properties, section History plugin configuration.

Changed properties

  • dgs.graphql.introspection.enabled - renamed to spring.graphql.schema.introspection.enabled, other dgs.graphql.* properties renamed to spring.graphql.*, see Migration from DGS to Spring GraphQL. After the upgrade, change the property names. Alternatively, remove those properties to use the default values.

Removed properties

nme-config.xml

Removed properties

  • History plugin configuration: move the configuration from the nme-config.xml file to the application.properties file, see History plugin configuration.

    <plugin class="com.ataccama.nme.history.NmeHistoryPlugin">
    <persistenceLayer class="com.ataccama.nme.persistence.vldb.VldbPersistenceFactory">
        <dataSource>mdc_db</dataSource>
        <prefix>H_</prefix>
    </persistenceLayer>
    <configFile>nme-history.gen.xml</configFile>
    <storageDirectory>../storage/history</storageDirectory>
</plugin>

runtime.parameters

Added properties

  • Matching step parallelism - see Matching step parallelism configuration:

    • nme.matching.partition.parallel

    • nme.matching.key.parallel

    • nme.matching.rule.parallel

    • nme.matching.parallel.threshold

    • nme.matching.parallel.group.threshold

    • nme.matching.parallel.statistics

  • Runtime hint parameters - see Runtime Parameters, section Hint parameters:

    • VldbTableAccessor.find.main.small

    • VldbTableAccessor.find.main.large

    • VldbTableAccessor.count.accuracy.mode

    • DbInList.main.hint

Was this page useful?