Initial Load Operation
A load operation represents a batch input interface in the MDM Hub solution for a given connected system. An initial load operation represents a type of batch input interface that is used for the first time data processing (that is, the system has never been loaded into the MDM hub yet).
To add a new initial load operation, follow these steps.
Prerequisites
-
Make sure the instance layer model is configured as described in Creating an Instance Layer Model using the top-down modeling approach (see Creating a Logical Model for more information about the modeling approaches).
-
Connect a system for which you want to add a batch load operation. See Connecting a System.
Add the load operation definition
-
Open the MDM project in ONE Desktop.
-
Open Connected Systems > [system] > Load Operations (right-click) and select Initial Load.
-
In the initial load configuration, provide the operation name suffix and choose to either build the load component from scratch or reuse the load component from the Full Load Operation by selecting Use Full Load Plan.
Reuse the component configured for the full load to save time. In addition, this brings some performance benefits. -
Optionally, set the Ignore Persistence flag (the empty default works in most cases).
This is an expert setting. -
Generate the load operation file and finish the implementation of the load plan as described in Full Load Operation. Skip the latter part of this step if you reused the full load plan.
Resumable initial load checkpoints
Activate initial load checkpoints so that, if the load operation fails, you can rerun it from the last successful consolidation action rather than from the beginning. MDM saves the first checkpoint at the start of Master Data Consolidation and then after every consolidation action (such as Clean, Match, Merge).
To activate this functionality, run the initial load with the parameter checkPointName defined.
For more information about the configuration, see Defining Load Parameters.
A checkpoint is a directory with files containing records.
By default, it is stored in mdc_checkpoint, which is in the temporary folder (tmpdir, this can be changed using the runtime parameter nme.checkPoint.directory).
The checkpoint folder is named after the value of checkPointName parameter (passed to the batch load operation).
It is automatically deleted upon successful completion of the load.
If the batch load fails, run it again using the same value of checkPointName parameter defined.
MDM then detects the checkpoint (that is, it locates the temporary folder) and skips Data Acquisition, Change Detection (or Data Preparation), and any consolidation actions finished in the previous run.
The load operation can fail and rerun multiple times, each time moving further to end.
| The checkpoint feature should be used only manually. Using it in automated workflows can cause the process to fail. |
| Do not edit data between load operation fails and reruns as Change Detection is skipped and you might lose some updates. |
nme.parallel.strategy must be set to none (default).
|
Monitoring load operation
You can check the status by viewing Task Properties on the Execution Status screen in the Admin Center.
For more information, see MD Process Monitoring.
Was this page useful?