Runtime Parameters
The following parameters are expert settings only. There is usually only an expert way to decide the right value and therefore there is no guarantee of improved performance when a parameter is used. |
To use some of these parameters, just add them as a java parameter, for example, -Dnme.copyColumns.skipUnchanged=true
or put them in the runtime.properties
file.
Runtime parameters values set during engine startup are available in ONE Runtime Server Admin.
Process Control
Name | Default Value | Description | ||||
---|---|---|---|---|---|---|
nme.copyColumns.skipUnchanged |
|
If
|
||||
nme.match.rematchIf.removeManualMatch |
|
If |
Performance
Name | Default Value | Description |
---|---|---|
nme.clean.parallelism |
|
Default parallelism setting for cleansing plans.
Can be overridden by |
nme.validate.parallelism |
|
Default parallelism setting for master validation. |
nme.multiLoad.parallel |
|
Number of delta load operations that will be executed in parallel during a MultiLoad operation. |
nme.delta.parallel |
|
Number of threads that will be used to load entities in delta load operations. |
nme.delta.parallel.threshold |
|
Minimum number of incoming records (all entities together) to trigger parallel processing of delta load operation (only if nme.delta.parallel > 1). |
nme.consolidation.parallel |
|
Number of threads (and database connections) that will be used in consolidation phase. |
nme.consolidation.parallel.threshold |
|
Minimum number of modified records (all entities together) after delta detection to trigger parallel consolidation (only if nme.consolidation.parallel > 1). |
nme.commit.parallel |
|
Number of threads (workers) available during the Committing phase. Workers can simultaneously insert entities into repository. |
nme.commit.parallel.threshold |
|
Minimum number of modified records (all entities together) to trigger parallel committing (only if nme.commit.parallel > 1). |
nme.commit.parallel.entity.maxWorkers |
|
Maximum number of threads (workers) assignable to an entity during the committing phase. Enables parallel committing to a single entity. |
nme.commit.parallel.sort |
|
A metric to prioritize entities for parallel committing. Possible values:
|
nme.commit.trimTooLongStrings |
|
If |
nme.taskExecutor.parallel |
|
Number of tasks (loads, exports) being executed by Task Executor in parallel (as long as synchronization conditions allow that). |
nme.export.read.parallel |
|
The number of data sources that will be read in parallel in a single export operation. |
nme.export.read.parallel.nameFilter |
\ |
List of comma-separated names of export operations that will support parallel retrieval of its data sources (asterisk wildcard is supported). |
nme.io.storage.memSize |
|
Number of records kept in memory in temporary storages. Any excessive records are written to the hard disk and deserialized when needed. This is a shared parameter and will be applied to each entity from the Instance & Master models. |
nme.io.storage.compress |
|
Temporary files storing processed records can be compressed. Possible values:
|
nme.checkPoint.directory |
same as |
Base folder to store MDM checkpoint data for a resumable initial load operation.
Keep in mind that the parameter can display NULL value if not specified by a user, however, internally it still uses the same base folder as defined in |
nme.checkPoint.restore.parallel |
|
The number of threads that will be used to restore the checkpoint. |
nme.consolidation.pool.minSize |
|
Minimum size of the pool of plan runners for Cleanse, Match and Merge plans. |
nme.consolidation.pool.maxSize |
|
Maximum size of the pool of plan runners for Cleanse, Match and Merge plans. |
nme.parallel.strategy |
|
Parallelism mode of RW operations:
|
nme.parallel.coordinator.key.strategy |
|
Strategy for key processing to optimize memory usage when large source IDs are used. Possible values:
|
Other
Name | Default Value | Description |
---|---|---|
nme.taskExecutor.taskLogLevel |
|
By default start and finish of all operations (load, export, reprocess, processDelta) and their subtasks are logged to standard logger (usually StdOut). This can be switched off by setting this parameter to
|
nme.taskExecutor.taskPersistLevel |
|
If set to If set to |
nme.taskExecutor.debug |
|
If |
nme.statistics.maxDirtyCount |
|
Data statistics are computed in incremental way, creating partial counts that needs to be summed to get a correct total value. If the number of partial counts is greater than this parameter, partial counts are summed and replaced by a single total count. |
nme.eventHandler.active |
|
If If |
nme.eventHandler.publish.batchSize |
|
Data change events are (for performance reasons) published in batches, this parameter sets the number of events in one batch. |
nme.stream.consumers.active |
|
Stream Consumers are stopped on server start. This option allows starting all stream consumers on server startup. |
nme.services.range.maxCount |
|
Maximum allowed value of count element in range expression in list and search services. If range expression is not given in request it is the default value. |
nme.services.values.returnNulls |
|
If |
nme.tmpStorage.checkDuplicates |
|
If |
nme.normalize.emptyStringIsNull |
|
If
The parameter is set to |
nme.model.source_id.length |
|
Size of the source_id column. Keep in mind that this column is indexed, and some databases (Sybase ASE, MySQL) have a size limit for indexable varchar columns, which that setting this parameter to a large enough value might result in a "max index size breached" error. |
nme.tasks.migration.apply-automatically |
|
If set to |
DB / Storage
VLDB Persistence Parameters
The following parameters are usable only when VLDB Persistence is used (see Persistence Layer, section VLDB Persistence).
Name | Default Value | Description |
---|---|---|
nme.vldb.inList.batchSize |
|
Batch size for inList input data. |
nme.vldb.inList.commitSize |
|
Commit size for inList input data. |
nme.vldb.batchSize |
|
Batch size for writing data in the Committing phase. |
nme.vldb.commitSize |
|
Commit size for writing data in the Committing phase. |
nme.vldb.readonly |
|
If In read-only mode, the transactions cannot change the data in repository.
In cluster installations (several NME instances are running over same DB storage), this flag should be set to |
nme.vldb.smallDataThreshold |
|
Threshold for number of inList records determining if LARGE or SMALL data hints should be used. |
nme.vldb.tinyDataThreshold |
|
Threshold for number of inList records, numbers below are considered tiny and different access method is used - direct access instead of inList. |
nme.vldb.prefetch.large |
/ |
If set, the value of this argument will be used as prefetch size for SELECT statements for LARGE expected data sizes. |
nme.vldb.prefetch.small |
/ |
If set, the value of this argument will be used as prefetch size for SELECT statements for SMALL expected data sizes. |
nme.vldb.alterTableLevel |
|
Defines what alter table commands will be executed if a table in database does not match the model. This is list of permitted values:
|
nme.vldb.visibilityResolver.tmpTableThreshold |
|
Threshold for number of Logical Transactions present in the TREG to switch engine queries using global temporary tables. |
nme.vldb.xidMemory.size |
|
Number of records in RW transactions (XIDs) that are stored on transaction for faster LTC - per one transaction and accessor (entity). |
nme.vldb.xidMemory.totalSize |
|
Number of records in RW transactions (XIDs) that are stored for faster LTC - total maximum. |
Unification Extended Hint Parameter
The following parameter is usable only when VLDB Persistence (see Persistence Layer, section VLDB Persistence) and Extended Unification step are used.
Name | Default Value | Description |
---|---|---|
nmeRepositoryCommandsFile |
/ |
Defines Extended Unification hints path to repository_nme.xml file (default location is runtime/repository_nme.xml). |
Data Collector (LTC) Parameters
The following settings belong to an LTC feature related to the VLDB Persistence (see Persistence Layer, section VLDB Persistence). The LTC stands for Logical Transaction Collector, and it is responsible for deleting obsolete records (for example similar to Java garbage collector but dealing with obsolete transactions' data).
Name | Default Value | Description |
---|---|---|
nme.vldb.collector.enabled |
|
If |
nme.vldb.collector.batchSize |
|
Batch size (#deleted records in one execute() call) for the LTC process. |
nme.vldb.collector.batchWait |
|
Delay (in ms) between executing individual deletion batches in the LTC process. |
nme.vldb.collector.commitSize |
|
Commit size (#deleted records in deletion commit sets) for the LTC process. |
nme.vldb.collector.commitWait |
|
Delay (in ms) between processing individual deletion commit sets in the LTC process. |
Hint Parameters
The parameters below make it possible to apply hints for some special computations being done during the processing.
The following hint examples are usable for Oracle 11+ database only. |
Name | Default Value | Description | ||
---|---|---|---|---|
VldbUnorderedInList.match.small |
|
Hint for retrieving matched records for SMALL inLists in unordered lists. |
||
VldbUnorderedInList.match.large |
|
Hint for retrieving matched records for LARGE inLists in unordered lists. |
||
VldbUnorderedInList.unmatch |
`/*+ORDERED USE_HASH(l d x) FULL(l) FULL(d) FULL(x) SWAP_JOIN_INPUTS(l) SWAP_JOIN_INPUTS(x) */ ` |
Hint for retrieving unmatched records for inLists in unordered lists.
|
||
VldbOrderedInList.match.small |
|
Hint for retrieving matched records for SMALL inLists in ordered lists. |
||
VldbOrderedInList.match.large |
|
Hint for retrieving matched records for LARGE inLists in ordered lists. |
||
VldbOrderedInList.unmatch |
`/*+ORDERED USE_HASH(l d x) FULL(l) FULL(d) FULL(x) SWAP_JOIN_INPUTS(l) SWAP_JOIN_INPUTS(x) */ ` |
Hint for retrieving unmatched records for inLists in ordered lists.
|
||
VldbTableAccessor.find.small |
|
Hint for retrieving data via SELECT WHERE for SMALL datasets. |
||
VldbTableAccessor.find.large |
|
Hint for retrieving data via SELECT WHERE for LARGE datasets. |
DB Persistence Parameters
The following parameters are usable only when DB Persistence (Persistence Layer, section DB Persistence) is used.
Name | Default Value | Description |
---|---|---|
nme.db.inList.batchSize |
|
Batch size for inList input data. |
nme.db.inList.commitSize |
|
Commit size for inList input data. |
nme.db.batchSize |
|
Batch size for writing data in the Committing phase. |
nme.db.commitSize |
|
Commit size for writing data in the Committing phase. |
nme.db.smallDataThreshold |
|
Threshold for the number of records deciding if LARGE or SMALL data hints should be used. |
nme.db.tinyDataThreshold |
|
Threshold for number of records, numbers below are considered tiny and different access method is used - direct access instead of inList. |
nme.db.alterTableLevel |
|
Defines what alter table commands will be executed if a table in database does not match the model. This is list of values, permitted values:
|
Hint Parameters
The parameters below make it possible to apply hints for some special computations being done during the processing.
The following hint examples are usable for Oracle 11+ database only. |
Name | Default Value | Description |
---|---|---|
DbUnorderedInList.match |
/*+ORDERED USE_HASH(l d) SWAP_JOIN_INPUTS(l) FULL(l) FULL(d) */ |
Hint for retrieving matched records for inLists in unordered lists. |
DbUnorderedInList.unmatch |
/*+ORDERED USE_HASH(l d) SWAP_JOIN_INPUTS(l) FULL(l) FULL(d) */ |
Hint for retrieving unmatched records for inLists in unordered lists. |
DbOrderedInList.match |
/*+ORDERED USE_HASH(l d) SWAP_JOIN_INPUTS(l) FULL(l) FULL(d) */ |
Hint for retrieving matched records for inLists in ordered lists. |
DbOrderedInList.unmatch |
/*+ORDERED USE_HASH(l d) SWAP_JOIN_INPUTS(l) FULL(l) FULL(d) */ |
Hint for retrieving unmatched records for inLists in ordered lists. |
Was this page useful?