Complex Delta Load Operation
A load operation represents a batch input interface in the MDM Hub solution for a given connected system. A complex delta load operation represents a type of batch input interface that is used for processing of partial data (for example, set of daily changes - increments).
To add a new complex delta load operation, follow these steps.
Prerequisites
-
Make sure the instance layer model is configured as described in Creating an Instance Layer Model using the top-down modeling approach (see Creating a Logical Model for more information about the modeling approaches).
-
Connect a system for which you want to add a batch load operation. See Connecting a System.
Add the load operation definition
-
Open the MDM project in ONE Desktop.
-
Open Connected Systems > [system] > Load Operations (right-click) and select Complex Delta Load.
-
In configuration, provide the operation name suffix.
-
Under Selected Entities, list the processed entities. If required, limit the columns (Selected Columns) affected by this configuration for all necessary entities.
-
Set up Delta Dependency Mode for each entity. This provides the instructions to the engine about how to detect changes when processing an input file in the delta load and which foreign key column to use. The engine expects an additional column in the input interface denoting the type of record change (change_type:
I- insert,U- update,D- delete).In hierarchical formats like XML or JSON, the records are typically provided with change type on the root element wrapping the central entity and its dependent entities that inherit the change type.
In flat file formats like CSV, the interface is typically composed of multiple files, each containing a change type column.
-
Define Ignored Comparison Columns and Partitioned Entities on the Advanced tab if needed.
-
Right-click the new load operation and select Generate. This prepares a ONE plan with predefined mapping and interface steps.
Once you have defined the load operation interface, you can configure the generated plan.
Configure the load plan
Finish implementing the load plan as described in Full Load Operation.
Was this page useful?