MDM Integration Architecture
MDM can work in online, stream, batch, as well as hybrid mode, supporting all the integration tasks behind those different deployments.
| For implementation details, see MDM Integration. |
Services
On a SOA environment, MDM offers the following:
-
Native services: A set of out-of-the-box services for common MDM operation and other services, such as maintenance, monitoring, and supporting operations. These services are derived from the model built as the core of the MDM project.
Supported types of services are SOAP over HTTP, XML over HTTP, and XML over JMS. All services are also available as JSON using REST API.
-
Custom services: Services created and maintained by developers. All created processes can be exported as online services.
The tool supports SOAP over HTTP, JMS, and JSON services out-of-the-box, and custom CSV over HTTP service to allow for bulk processing with minimal overhead.
Developers can also create compound services by integrating MDM with an ESB and building complex composite services. Those services can be composed using both native and custom services.
Streaming and message queues
MDM provides inbound interfaces to consume data from streaming or message queues. It also provides data events for further processing.
-
Stream Interface: Allows reading messages from a queue using JMS or native connectivity and transforms messages into MDM entries. Messages in the queue are processed after the queue collects the defined maximum number of messages or after the defined maximum waiting time. The processing starts whenever one of the parameters is satisfied.
Supports AWS SQS, Kafka, Confluent Kafka, Salesforce, and any JMS-enabled message queue.
-
Event Handler and Event Publisher: Capture certain user-defined changes during data processing. Multiple event handlers can be defined at once.
-
Event handlers are able to function in the asynchronous mode and support filtering to select only certain events. They can also trigger a publishing event.
-
Event publishers can transform data in a plan and send it to a selected consumer using standard connectivity as database (see Database systems), JMS Writer, Kafka Writer, SOAP or REST API calls.
The Event Handler stores the events and therefore requires either a dedicated database schema or available persistent memory to be used (that is, a persistent filesystem).
-
File formats
MDM contains fully customizable reader and writer steps for both delimited and fixed-width text files. These files can be encoded in as many formats as supported by the Java technology. MDM is able to understand both the encoding and the structure and parse the data from the files adequately.
Various industry-adopted binary formats (such as MS Excel or DBF) are supported:
-
Microsoft Excel readers and writers support multiple versions as well as Excel templates for creating formula-based and rich Excel files.
-
Support for reading different file formats. This is provided via the Component Library, where individual readers or adapters are developed on top of the MDM Generic Data Reader step. For example, DTAUS file, mainframe/COBOL binary files with Copybook metadata, and other file formats.
MDM supports integration with files that follow a structured standard, for example:
-
MDM can read and write to structured data files on XML. XML is the main standard used throughout the tool, and there is a special group of steps to both read and write data.
The data is flattened through a process of stream reading and deflattened by confectioning template structures, which are parametrized with the values coming from the records in the MDM flow. xPath is supported to access the data more efficiently.
-
MDM also supports other structured data types, such as JSON, in a similar fashion.
Database systems
MDM can connect to a wide array of database systems. This integration can be achieved in two different ways: natively or by using special connectors to particular legacy or non-RDBMS systems.
-
Connectivity to all industry-standard RDBMS engines is provided by JDBC and ODBC drivers. These can be for input and output operations, as sources or targets.
Out of the box, MDM includes JDBC drivers for most common enterprise databases, such as Oracle, MS SQL, Teradata, DB2, and PostgreSQL.
-
In addition, MDM can use special connectors or adapters that support all major relational and non-relational databases, as well as legacy systems, information systems, CRM/ERP systems, and more.
Connectivity to non-relational and legacy systems is maintained by the aforementioned adapters. These offer connectivity to over 300 systems, enabling access to virtually any other system.
Integration with other Ataccama products
MDM can connect to the following Ataccama applications to make the most out of the platform functionality.
- Ataccama ONE Data Quality & Catalog (DQ&C)
-
A bidirectional integration is available between MDM and DQ&C. If configured, MDM tables are visible in the Catalog, allowing you to measure data quality and enrich data structures using business terms. In turn, the assigned business terms, both on the table and attribute level, are shown in MDM Web App.
To learn how to set up this connection, see MDM and RDM Connection.
- ONE RDM
-
RDM can be used as a source of reference data as well as for mapping source system’s codes to master codes. The integration between RDM and MDM is typically done using RDM integration steps in MDM batch load operations.
Was this page useful?