Persistence Layer
The persistence layer stores model data to the persistent backend. This definition is part of the MDM configuration and is shared by all defined models, that is, shared tables are created in the specified database.
VLDB persistence
VLDB persistence (also known as the logical transaction persistence) is a database persistence that does not rely on database transactions and supports large change sets through an application transaction handling.
All data changes are realized as INSERT commands, with obsolete records being deleted over time (expunger job, also known as Logical Transaction Collector).
Major features supported by the persistence are listed in Available features by persistence type.
<persistenceLayers>
<persistenceLayer class="com.ataccama.nme.persistence.vldb.VldbPersistenceFactory">
<dataSource>MDC_DB</dataSource>
<prefix>A_</prefix>
<binding>true</binding>
<commandTemplatesFile>vldb-commands.xml</commandTemplatesFile>
</persistenceLayer>
</persistenceLayers>| The following optional settings affect the MDM engine performance. Use with caution. |
| Parameter | Default | Description |
|---|---|---|
|
true |
Setting the parameter to |
|
Oracle DB hints |
Path to a file with hinted database queries used by VLDB persistence. See Runtime Parameters for Oracle hint explanations and Database Hints for template file samples. |
Database persistence
Standard database persistence relying on database transactions. Usable for medium-sized implementations and large-size implementations where only a reasonable amount of data is changed in each load.
Major features supported by the persistence are listed in Available features by persistence type.
<persistenceLayers>
<persistenceLayer class="com.ataccama.nme.persistence.database.DbPersistenceFactory">
<dataSource>MDC_DB</dataSource>
<prefix>A_</prefix>
</persistenceLayer>
</persistenceLayers>Available features by persistence type
Choosing a proper type of persistence layer depends on the expected behavior of the MDM solution. The following is a list of supported features.
| Group | Feature | DB persistence | VLDB persistence | Notes |
|---|---|---|---|---|
Load |
Delta load |
Yes |
Yes |
/ |
Load |
Multiload |
Yes |
Yes |
Ability to load data from multiple systems in one batch. |
Load |
Streaming |
Yes |
Yes |
/ |
Services |
Read-only services |
Yes |
Yes |
/ |
Services |
Read-write services |
Yes |
Yes |
/ |
Export |
Batch export |
Yes |
Yes |
/ |
Export |
Event handler |
Yes |
Yes |
/ |
Parallelism |
Batch load |
Yes |
Yes |
Run multiple independent batch load operations. |
Parallelism |
Streaming |
Yes |
Yes |
Consume data from multiple independent stream input interfaces at once. |
Parallelism |
Online (RO) |
Yes |
Yes |
/ |
Parallelism |
Online (RW) |
Yes |
Yes |
Support multiple RW services calls concurrently, typically processDelta. |
Parallelism |
Export |
Yes |
Yes |
Run multiple independent batch export operations. |
Parallelism |
Batch load and online (RW) |
Yes |
Yes |
Run a batch load opearation with concurrent online RW services (for example, processDelta). |
Parallelism |
Batch load and export |
Yes |
Yes |
Run any batch load concurrently with any batch export. |
Parallelism |
Transaction |
DB |
LT |
Database transactions vs. NME logical (application) transaction. |
Parallelism |
Default transaction isolation level |
Read committed |
Repeatable read |
The setting always depends on the database setup (specifically, the transaction isolation level). |
Parallelism |
Reprocess only conflicting rows |
No |
Yes |
Restarting only conflicting records of a transaction vs. the entire transaction reload. |
Parallelism |
Parallel change detection |
No |
Yes |
Run parallel change detection for multiple entities during a load operation. |
Parallelism |
Parallel master data consolidation |
No |
Yes |
Ability to run parts of the NME iWF in parallel (for example, cleansing of two independent entities). |
Parallelism |
Elastic committing |
No |
Yes |
Ability to dynamically adjust the number of parallel threads performing the Commit phase. |
Parallelism |
Parallel data reading in export |
No |
Yes |
Data preparation (reading from MDM storage) can read multiple entities at once in export. |
Matching |
Matching step |
Yeses |
Yeses |
/ |
MDA |
MDA data storage |
Yes |
Yes |
VLDB is supported as MDA storage starting from 12.3.0. |
MDA |
Record drafts |
Yes |
Yes |
/ |
MDA |
Attribute overrides |
Yes |
Yes |
/ |
Optimization |
Customization of SQL commands via templates |
No |
Yes |
Adjustment of internal SQL statements for accessing MDM storage. |
Optimization |
Support SQL hints for small or large data sets |
No |
Yes |
Support different SQL statements (DB hints) for different sizes of data sets. |
Optimization |
Support for very small transactions |
No |
Yes |
Support for very small transactions (for example, web services) using |
Optimization |
Exported columns selection |
No |
Yes |
Retrieve only selected columns from the database, as defined in the export operation. |
Was this page useful?