ONE Desktop ONE DQG ONE MDM ONE RDM ONE Runtime Server
latest (15.1.0) 14.5.x 13.9.x 13.8.x 13.7.x 13.6.x 13.5.0 13.4.x
User Community Service Desk Downloads
If you can't find the product or version you're looking for, visit support.ataccama.com/downloads

Model

The MDM model is the definition of logical layers (instance and master), event handlers, and settings. Models, and their database connections and persistence types, are defined in nme-config.xml (see MDM Configuration).

nme-config.xml
<config>
    ...
    <model>
        <models>
            <consolidation>
                <persistenceLayer class="com.ataccama.nme.persistence.database.DbPersistenceFactory">
                    <dataSource>mdc_db</dataSource>
                    <prefix>C_</prefix>
                </persistenceLayer>
                <configFile>nme-model.gen.xml</configFile>
                <eventHandlersConfigFile>nme-event_handler.gen.xml</eventHandlersConfigFile>
                <executionPlanConfigFile>nme-consolidation-plan.xml</executionPlanConfigFile>
            </consolidation>
        </models>
    </model>
    ...
</config>

Configuration Parameters

A full model configuration consists of the following parameters:

Parameter Required Description

persistenceLayer

Y

The type of persistence layer to be used for this model (the class attribute). See Persistence Layer for persistence types.

persistenceLayer/dataSource

Y

The name of the database connection in runtime configuration (see Runtime Configuration).

persistenceLayer/prefix

N

This prefix will be added to all tables created for this model.

configFile

Y

The path to the main configuration file of the model. Contains the definition of the logical models: entities and their relationships, transformation operations, and other settings.

eventHandlersConfigFile

N

The path to the event handler configuration for this model type (see Event Handler). Each model type has a separate configuration.

executionPlanConfigFile

N

Optional, used only for the Consolidation model. The path to the execution plan (see Execution Plan).

Model Definition

The configured model - nme-model.gen.xml - is located in Files/etc. The file is generated when at least the instance layer is configured.

The main purpose is to define:

  1. Instance layer

    • instance entities and relationships among them

    • copy columns

    • operations

    • source system definition and mapping to MDM instance entities

  2. Master layers

    • master layer name

      • master entities and relationships

      • relationships among master and instance entities

      • entity behavior and operations

      • external entities and relationships

Basic structure of nme-model.gen.xml is the following:

<?xml version='1.0' encoding='UTF-8'?>
<model>
    <instanceLayer>
        <entities>
        ...
        </entities>
        <sourceSystems>
        </sourceSystems>
    </instanceLayer>
    <masterLayers>
        <masterLayer name="Master">
          <entities>
          ...
          </entities>
          <externalEntities>
          ...
          </externalEntities>
        </masterLayer>
        <masterLayer name="...">
        ...
        </masterLayer>
    </masterLayers>
</model>

As shown here, it is possible to define one instance layer including entities and source system definitions, and several master layers and their entities. The file also contains a link to the event handler configuration (see Event Handler).

Instance Layer

Instance Entities

Defines entities and their behavior on the instance layer. An entity is defined by columns, relationships and operations as shown in the following example.

<entities>
  <entity name="party">
    <overridable data="true" matching="true" activity="true"/>
    <columns>
      <column name="src_first_name" origin="source" type="string" size="100"/>
      <column name="std_first_name" origin="clean" type="string" size="100"/>
      ..
    </columns>
    <relationships/>
    <cleansingOperation>
    ..
    </cleansingOperation>
    <matchingOperation>
    ..
    </matchingOperation>
  <aggregatingOperation>
  ..
  </aggregatingOperation>
  </entity>
  <entity name="rd_gender" usage="DICTIONARY">
    <overridable data="true" matching="true" activity="true"/>
    <columns>
      <column name="master_code" origin="source" size="1000" type="string"/>
      <column name="master_name" origin="source" size="1000" type="string"/>
    </columns>
  </entity>
</entities>

Parameters:

  • name: the name of the entity.

  • usage: usage of the entity.

    • DATA_ENTITY: default, standard data entity.

    • DICTIONARY: entity with a relatively small number of records used for storing dictionary values, no indices will be generated on relationships from data entity to dictionary.

  • <overridable>: defines if data, matching, and activity overrides are enabled for the instance entities.

Column

Column definition for instance entities is done this way:

<columns>
   <column name="src_name" origin="source" type="string" size="100" overridable="true"/>
     <column name="std_name" origin="clean" type="string" size="100"/>
     <column name="src_ssn" origin="source" type="string" size="100" overridable="true"/>
     <column name="std_ssn" origin="clean" type="string" size="100"/>
     <column name="uni_can_id" origin="match" type="long"/>
   <column name="agg_group_count" origin="aggregate" type="string" size="1000"/>
</columns>

Parameters:

  • name: the name of the column.

  • origin (column definition context - instance columns only): defines a place in a plan where the column appears.

    • source: source value, appears in cleanse plan input.

    • clean: cleansed, appears as a cleanse plan output.

    • match: match value, appears as a match plan output.

    • aggregate: aggregation output, appears as an aggregation plan output.

  • type: data type.

  • size: column length (used as a database column size) that is mandatory for string data type.

  • overridable: specifies if data overrides are allowed on the column level.

Copy Columns

Copy column functionality is necessary for copying an attribute from one entity to another. To define a copy column, there must exist a relationship definition between the entities. After it is defined, it is possible to define the direction of copy columns: parent to child or child to parent.

<column name="mrg_addresses" origin="COPY_CLEAN" type="STRING" size="100">
  <valueSource relationshipName="addresses" columnName="std_value">
    <aggregation class="com.ataccama.nme.engine.model.ConcatenateDistinct" separator=";" maxLength="100"/>
    <filterExpression>source.std_address_type = "R"</filterExpression>
  </valueSource>
</column>

Parameters:

  • name, type and size: see the column definition example in this section.

  • origin (copy column context): defines which columns are taken and where the copied columns appear.

    • copy_source: a source column added to a cleansing plan output.

    • copy_clean: a cleansed result column added to a match plan input.

    • copy_match: a matched result column added to a merge plan input.

  • valueSource:

    • columnName: the source column for the copy.

    • relationshipName: the relationship definition and the direction to be used for the copy.

  • aggregation class:

    • ConcatenateDistinct: used to perform child to parent copy column. Because of the 1:M relationship, there is a concatenate distinct functionality to store all the values related to a single attribute on parent entity.

      • separator: the values separator that is used for concatenation.

      • maxLength: the length of the concatenated outcome.

    • MinValue : used to perform child to parent copy column. It takes the minimum value from all related entities (because of 1:M relationship).

    • MaxValue : used to perform child to parent copy column. It takes the maximum value from all related entities (because of 1:M relationship).

    • FirstValue : used to perform parent to child copy column. It might be used for child to parent copy column, but there is no deterministic selection.

  • isFk (boolean): if true, this copy column is used as a foreign key for this relationship.

  • expression: the expression that is used to fill copy columns.

  • <filterExpression>(boolean): optional; provides an ability to determine when column copying should be performed using a filter expression. There are two pseudoinputs available: source (origin of copy) and target.

History Collector

Enables to keep historical values of a given column and stores concatenated values to a newly defined column. Such functionality might be useful in a match process.

<column name="std_uir_adr_id_h" origin="CLEAN" type="STRING" size="40000">
    <historyCollector columnName="std_uir_adr_id" separator="^~" maxCount="20"/>
</column>

Parameters:

  • columnName: the name of the source column.

  • separator: the values separator definition for concatenation.

  • maxCount: the number of values kept in history.

OldValueProvider

OldValueProvider is a way to access old value of instanceEntity column in transition plan.

<column name="std_expiration_date" origin="clean" type="datetime" size="100">
  <oldValueProvider columnName="std_expiration_date_old" />
</column>

Parameters:

  • columnName: defines the target column with oldValue, which is then available in transition plans.

oldValueProvider can be defined for columns with following origins:

Origin of column oldValue column is available in transition plan

SOURCE

clean, match, merge

COPY_SOURCE

clean, match, merge

CLEAN

clean, match, merge

COPY_CLEAN

match, merge

MATCH

match, merge

COPY_MATCH

merge

Do not use oldValue-based expressions after the Extended Unification step in a match plan. Some additional records might be expanded from unification repository during the match process and result will not be exact.

Instead, move this computation to the merge plan, where all the relevant data and their oldValues are available.

Instance Relationship

Relationship definition is always a part of a child entity definition and it points to a parent entity.

<relationships>
  <rel reverseName="addresses" parentEntity="party" parentColumn="source_id" name="party" foreignKeyColumn="pty_source_id">
      <type class="com.ataccama.nme.engine.model.SameSystemRelationship"/>
  </rel>
</relationships>

Parameters:

  • rel: main relationship definition with the following attributes.

    • name: relationship name; used as a reference name for copy columns.

    • reverseName: reverse relationship name; used as a reference name for copy columns.

    • parentEntity: name of the parent entity.

    • parentColumn: primary key column of the parent entity.

    • foreignKeyColumn: foreign key column of the child entity.

  • type: type of relationship defined in the class attribute.

    • SameSystemRelationship (default): relationship is created only between identifiers having the same origin, that is, within a single system.

    • CrossSystemRelationship: enables to create a relationship between two entities regardless the origin. Useful for code books and dictionaries. Same definition as an instance relationship, except type is not defined here since there is no longer a source system reference. Therefore, the behavior is the same as described for SameSystemRelationship.

    • SameSystemExtendedRelationship: relationship is created only between identifiers having the same origin, that is, within a single system. However, it is possible to define where information about record origin is stored (for the purpose of evaluating this relationship only):

      • record source system might be stored in any column (type string, column origin source) as source system name (same name as defined in source system definition), for example, "crm".

      • either parentSystemColumnName or childSystemColumnName or both must be defined:

        • parentSystemColumnName: name of column in the parent entity containing source system name of the parent record.

        • childSystemColumnName: name of column in the child entity containing source system name of the child record.

      • if the defined column contains null, the record origin is used; if defined column contains string that does not match any of the defined source systems, an exception is thrown (during the MDM consolidation process in CopyColumns action, during web service call with preloadedRelationships or traversal)

<relationships>
  <rel reverseName="related_parties"
     parentEntity="party"
     parentColumn="id"
     name="parent_party"
     foreignKeyColumn="parent_pty_id">
    <type class="com.ataccama.nme.engine.model.SameSystemExtendedRelationship">
        <parentSystemColumnName>src_party_system</parentSystemColumnName>
        <childSystemColumnName>src_parent_pty_system</childSystemColumnName>
    </type>
  </rel>
</relationships>

Instance Operation

Cleanse Operation

Defines a cleanse logic for a given entity which is represented by a ONE plan.

<cleansingOperation class="com.ataccama.nme.dqc.operations.CleansingPlan" planFileName="../trans/party/party_clean.comp"/>
Match Operation

Defines a match logic for a given entity, which is represented by a ONE plan. Currently there is only one implementation of matchingOperation available and it is compatible only with the Matching step.

<matchingOperation planFileName="../engine/trans/party/party_match.comp" class="com.ataccama.nme.dqc.mdu.MduMatchingPlan"/>
Multiple Matching steps in one matching plan

It is possible to have multiple Matching steps in one matching plan to achieve some extremely complex matching scenarios. In such case every Matching step must have filled different Repository Name and should have different bindings. Moreover, the matching plan itself requires more configuration:

<matchingOperation planFileName="../engine/trans/party/party_match.comp" class="com.ataccama.nme.dqc.mdu.MduMatchingPlan">
  <matchings>
    <!-- One matching element for every Matching step in plan. Name here must refer to Repository Name in step. -->
    <matching name="nm">
      <masterIdColumn>master_id_nm</masterIdColumn>
      <matchRuleNameColumn>match_rule_name_nm</matchRuleNameColumn>
      <matchQualityColumn>match_quality_nm</matchQualityColumn>
      <matchRelatedIdColumn>match_related_id_nm</matchRelatedIdColumn>
      <isolateFlagColumn>isolate_flag_nm</isolateFlagColumn>
      <rematchFlagColumn>rematch_flag_nm</rematchFlagColumn>
      <proposals enabled="true"/>
  </matching>
    <matching name="sin">
      <masterIdColumn>master_id_sin</masterIdColumn>
      <matchRuleNameColumn>match_rule_name_sin</matchRuleNameColumn>
      <matchQualityColumn>match_quality_sin</matchQualityColumn>
      <matchRelatedIdColumn>match_related_id_sin</matchRelatedIdColumn>
      <isolateFlagColumn>isolate_flag_sin</isolateFlagColumn>
      <rematchFlagColumn>rematch_flag_sin</rematchFlagColumn>
      <proposals enabled="false"/>
    </matching>
  </matchings>
</matchingOperation>

Every Matching step must have a corresponding definition in the matching operation in the model configuration file. The name of the matching must refer to the step using its Repository Name. Key column bindings must be configured for every matching:

Name in the Model Configuration File Name in the Matching Step Type Default column name Description

masterIdColumn

Master Id Column

LONG_INTEGER

master_id

Where the master group identifier is stored.

matchRuleNameColumn

Match Rule Name Column

STRING, 100

match_rule_name

Name of the matching rule that matched this record to another or null if the record is the only one in the master group.

matchQualityColumn

Match Quality Column

FLOAT

match_quality

Quality of match, 0 being the best, computed from matching tests (such as editDistance), if a simple Boolean expression is used, 0 represents a match.

matchRelatedIdColumn

Match Related Id Column

LONG_INTEGER

match_related_id

If a record is matched to another record, its instance record identifier is stored here.

isolateFlagColumn

Isolate Flag

BOOLEAN

isolate_flag

True if a record is isolated from matching with other records.

rematchFlagColumn

Rematch

BOOLEAN

rematch_flag

Not persisted. If true, the record is rematched and might change its master group identifier.

rematchIfChangedColumns

rematch If Changed Columns

BOOLEAN

/

Defines columns (<col>) in which changes will trigger rematch in all (or selected) matching steps.

proposals

Proposals

BOOLEAN

/

If enabled, another integration output in the matching plan will be generated to accept the pairs of proposed records (based on the defined rules).

These columns must not be present in the entity as the engine adds them automatically with correct types and origin. If there is only one Matching step, default values are used and no configuration is necessary. Default matching name (and Repository Name) is k.

Matching proposals and AI Matching

Defines entities that are used for working with matching proposals and AI Matching.

<proposals enabled="true">
  <defaultMasterEntityView view="masters" name="party"/>
  <entitiesForAiMatching>
    <entityForAiMatching view="norm" name="person"/>
    <entityForAiMatching view="norm" name="organization"/>
  </entitiesForAiMatching>
</proposals>

Parameters:

  • proposals:

    • enabled: enables matching proposals.

  • defaultMasterEntity: If specified, proposals will be accessible in MDM Web App. The values should correspond with matching operation.

    • view: name of the master layer.

    • name: name of the entity.

  • entitiesForAiMatching: optional list of entities for AI Matching.

    • view: name of the master layer.

    • name: name of the entity.

Aggregate Operation

An optional operation that defines aggregation logic for a given entity, which is represented by a ONE plan.

<aggregatingOperation class="com.ataccama.nme.dqc.operations.AggregatingPlan" planFileName="../engine/trans/party/party_aggregate.comp" groupingColumn="master_id" />

Parameters:

  • groupingColumn: specifies the grouping column used to collect all needed records into a group of records being processed. Column type has to be one of LONG_INT, INTEGER, LONG, STRING.

Source System Definition

Defines a list of source systems which are connected to the MDM engine.

<sourceSystems>
     <system name="crm">
          <instanceMappings>
                <originId name="crm_customer#Party" entity="party"/>
                <originId name="crm_address#Address" entity="address"/>
          </instanceMappings>
      </system>
      <system name="life">
          <instanceMappings>
                <originId name="life_contract#Party" entity="party"/>
                <originId name="life_contract#Address" entity="address"/>
          </instanceMappings>
       </system>
</sourceSystems>

Parameters:

  • <system name>: the source system name. It is available in eng_source_system column in ONE plans.

  • <instanceMappings>

    • name: a descriptive name that provides text information about record origin. It is available as eng_origin attribute in ONE plans.

      • <source_system>_<source_entity>#<target_MDC_entity>, for example, life_contract#Party.

      • <source_company>##<source_system>##<source_entity>, for example, ATACCAMA##LIFE##contracta.

Master Layers

On master layers, there are two types of entities: master entities (<entities>) and external entities (<externalEntities>).

Master Entities

Defines master entities and their behavior on the master layer. A master entity can behave in one of the defined ways: consolidation, authoring, or mixed (both options enabled at the same time).

  • Consolidation: records are read from external systems, preprocessed on instance layer and a master record is created based on the merge operation.

  • Authoring: records can be created on the master layer without having an instance layer available.

  • Mixed: the combination of consolidation and authoring.

Depending on the master entity behavior type, specific entity settings (that is, attributes and operations that can be performed) vary.

<entities>
  <entity class="com.ataccama.nme.engine.model.PersistentMasterEntity" dedicatedIds="false" name="party">
    <columns>
    ..
    </columns>
    <relationships/>
    <consolidation instanceEntity="party" groupingColumn="master_id">
      <overridable data="true" activity="true"/>
      <mergingOperation>
      ..
      </mergingOperation>
      <overrides>
      ..
      </overrides>
    </consolidation>
    <authoring>
      <masterMatching>
      ..
      </masterMatching>
    </authoring>
  </entity>
  <entity class="com.ataccama.nme.engine.model.VirtualInstanceEntity" instanceEntity="party" name="party_instance" >
    <relationships>
      ..
    </relationships>
  </entity>
</entities>

On the master layer, entities can be of two class types:

  • PersistentMasterEntity: defines a physical table to be stored in a database.

  • VirtualInstanceEntity: defines a virtual entity which is required to create a master-to-instance relationship.

Parameters:

  • name: the name of the entity.

  • usage (only for PersistentMasterEntity): specifies usage of the entity.

    • DATA_ENTITY: a default, standard data entity.

    • DICTIONARY: an entity with a relatively small number of records used for storing dictionary values, no indices will be generated on relationships from data entity to dictionary.

  • dedicatedIds (only for PersistentMasterEntity): specifies whether a certain entity uses its own sequence for the master_id and not from the shared sequence across the whole MDM solution. This is especially useful when there are limitations on the consuming system.

  • <consolidation>: enables consolidated records.

    • instanceEntity: a corresponding entity on the instance layer.

    • groupingColumn (only for PersistentMasterEntity): specifies the grouping column used to create a golden record. Column has to be of LONG_INT data type. Default column name is master_id.

  • <authoring>: enables authored records.

Column

Column definition for master entities is done this way:

<columns>
  <column name="cmo_first_name" type="string" size="100"/>
  <column name="cmo_last_name" type="string" size="100"/>
</columns>

Parameters:

  • name: the name of the column.

  • origin (column definition context - instance columns only): defines a place in a plan where the column appears.

    • source: a source value, appears in cleanse plan input.

    • clean: cleansed, appears as a cleanse plan output.

    • match: a match value, appears as a match plan output.

    • aggregate: an aggregation output, appears as an aggregation plan output

  • type: the data type.

  • size: column length (used as a database column size) that is mandatory for string data type.

  • overridable: data overrides on column level.

Copy Columns

<column name="cmo_contact_comp" origin="copy_merge" type="string" size="4000">
    <valueSource columnName="cmo_contact_comp" relationshipName="contacts">
        <aggregation class="com.ataccama.nme.engine.model.ConcatenateDistinct" maxLength="4000" separator="^~"/>
    </valueSource>
</column>

Parameters:

  • name, type and size: see the column definition example in this section.

  • origin (copy column context): defines which columns are taken and a place, where the copied columns appear.

    • copy_merge: source column added to a matching plan input.

  • valueSource

    • columnName: source column for the copy.

    • relationshipName: relationship definition and direction to be used for the copy.

  • aggregation class:

    • ConcatenateDistinct: used to perform child to parent copy column. Because the 1:M relationship, there is a concatenate distinct functionality to store all the values related to a single attribute on parent entity.

      • separator: values separator definition for concatenation.

      • maxLength: length of concatenated outcome.

    • MinValue: used to perform child to parent copy column. It takes minimum value from all related entities (because of 1:M relationship).

    • MaxValue: used to perform child to parent copy column. It takes maximum value from all related entities (because of 1:M relationship).

    • FirstValue: used to perform parent to child copy column. It might be used for child to parent copy column, but there is no deterministic selection.

  • isFk: (Boolean) if true, this copy column will be used as the foreign key for this relationship.

  • expression: the expression that is used to fill copy columns.

  • <filterExpression>: (boolean) optional; provides an ability determine when column copying should be performed using a filter expression. There are two pseudoinputs available: source (origin of copy) and target to construct the expression.

Master Relationship

Relationship definition is always a part of a child entity definition and it points to a parent entity.

<relationships>
  <rel foreignKeyColumn="master_id" name="master" parentColumn="id" parentEntity="contact" reverseName="instances"/>
    <rel foreignKeyColumn="src_type" name="fk_dic_type_contact" parentColumn="source_code" parentEntity="rd_cont_type_trans" reverseName="rev_fk_dic_type_contact">
      <type class="com.ataccama.nme.engine.model.SameSystemExtendedRelationship">
        <parentSystemColumnName>source_system</parentSystemColumnName>
      </type>
  </rel>
  <rel foreignKeyColumn="std_type" name="std_type_cont_type_master_details" parentColumn="master_code" parentEntity="rd_cont_type" reverseName="master_codes_std_type_cont_type_contact"/>
</relationships>

Parameters:

  • rel: the main relationship definition with the following attributes.

    • name: the name of the relationship; used as a reference name for copy columns.

    • reverseName: reverse relationship name; used as a reference name for copy columns.

    • parentEntity: the name of the parent entity.

    • parentColumn: the primary key column of the parent entity.

    • foreignKeyColumn: the foreign key column of the child entity.

  • type: type of relationship defined in the class attribute.

    • CrossSystemRelationship: has to be used for a master-to-master relationship and master-to-instance relationship. It might be used between VirtualInstanceEntity entities especially for code books and dictionaries. If <type> is not defined, CrossSystemRelationship is the default behavior of the relationship where one of the entities is PersistentMasterEntity.

    • SameSystemRelationship: can only be used between VirtualInstanceEntity entities. If type is not defined, SameSystemRelationship is the default behavior of the relationship between VirtualInstanceEntity entities.

    • SameSystemExtendedRelationship: can only be used between VirtualInstanceEntity entities.

For` PersistentMasterEntity` relatedRecordBehavior can be defined. It specifies how a change in the parent record affects its child records. For more information, see Mixed Style in MDM, section Related Record Behavior.

  • onAuthoredParentDelete: specifies options that can be enabled when an authored parent record is deleted.

    • deleteAuthoredRecord

    • followSurvivor

    • loseRecordOwnership

  • onParentLoseOwnership: specifies options that can be enabled when an authored parent loses its authored status.

    • deleteAuthoredRecord

    • loseRecordOwnership

  • onConsolidatedParentDelete: specifies options that can be enabled when a consolidated parent record is deleted due to matching.

    • allInstancesDeleted

    • followerExist

    • followerMissing

<relationships>
  <rel foreignKeyColumn="party_id" name="party" parentColumn="id" parentEntity="party" reverseName="notes">
    <relatedRecordBehavior>
      <onAuthoredParentDelete deleteAuthoredRecord="true" followSurvivor="true" loseRecordOwnership="true"/>
      <onParentLoseOwnership deleteAuthoredRecord="false" loseRecordOwnership="false"/>
      <onConsolidatedParentDelete allInstancesDeleted="KEEP" followerExist="FOLLOW" followerMissing="KEEP"/>
    </relatedRecordBehavior>
  </rel>
</relationships>

Validate Operation

Validation layer for the master data represented by a ONE plan.

<validatingOperation class="com.ataccama.nme.dqc.operations.ValidatingPlan" planFileName="../engine/trans/party/masters_party_validate.comp"/>

Operations and other settings available vary depending on the behavior of master entity: consolidation, authored, or mixed.

Consolidation Settings

Consolidation settings that can be defined are as follows:

<consolidation groupingColumn="master_id" instanceEntity="party">
    <overridable activity="true" data="true"/>
    <mergingOperation class="com.ataccama.nme.dqc.operations.MergingPlan" customActivityTracking="false" planFileName="../engine/trans/party/masters_party_merge.comp"/>
    <overrides>
        <enable>true</enable>
        <lifeCycle class="com.ataccama.nme.engine.model.lifecycle.RemoveOverrides">
            <columns>
                <column name="cmo_first_name" ifSame="true"/>
                <column name="cmo_last_name" ifSame="true"/>
                <column name="cmo_birth_date" ifSame="true"/>
            </columns>
        </lifeCycle>
    </overrides>
</consolidation>

Parameters:

  • <overridable>: defines if data and activity overrides are enabled or disabled.

Merge Operation

Operation defines a link between an MDM plan representing a merge operation and master entity definition.

Parameters:

  • customActivityTracking (Boolean): use custom rules for activity calculation, that is, overwrite eng_active value in the merge plan. false by default.

  • <recordFilterExpression> (Boolean, optional): enables to apply a filter to the merge plan record input. The expression cannot contain the sequence() function.

Overrides Lifecycle

The parameter defines if overrides are removed automatically and upon which conditions. The <enable> parameter enables overrides:

  • true: overrides are considered 'temporary' and are removed automatically if certain conditions are met.

  • false: overrides are considered 'permanent' and can only be removed manually.

Parameters:

  • class: when overrides are enabled, the class is RemoveOverrides.

  • <column>:

    • name: the name of the column whose values are compared to the incoming values.

    • types of conditions under which overrides are removed automatically:

      • ifSame: if the override value is equal to the incoming value (regardless of the original value).

      • ifChanged: if the original value is different to the incoming value (regardless of the override value).

      • expression: if an expression condition is satisfied.

Authored Settings

Authored settings that can be defined are as follows:

<authoring>
    <masterMatching class="com.ataccama.nme.engine.model.NmeMasterMatchingWithConsolidation">
        <matchingColumns>
            <column name="mat_party_type" expression="upper(cmo_type)"/>
            <column name="mat_gender" expression="upper(cmo_gender)"/>
            <column name="mat_first_name" expression="upper(squeezeSpaces(cmo_first_name))"/>
            <column name="mat_last_name" expression="upper(squeezeSpaces(cmo_last_name))"/>
            <column name="mat_full_name" expression="upper(squeezeSpaces(cmo_first_name + ' ' + cmo_last_name))"/>
            <column name="mat_initials" expression="upper(squeezeSpaces(sortWords(set.mapExp(cmo_first_name + ' ' + cmo_last_name, ' ', (x) { left(x,1) }))))"/>
            <column name="mat_person_id" expression="upper(squeezeSpaces(cmo_pid))"/>
            <column name="mat_birth_date" expression="cmo_birth_date"/>
            <column name="mat_contact_set" expression="upper(cmo_contact_comp)"/>
        </matchingColumns>
    </masterMatching>
    <lifeCycle class="com.ataccama.nme.engine.model.lifecycle.LoseOwnership">
        <conditions>
            <condition class="com.ataccama.nme.engine.model.lifecycle.ExpressionCondition">
                <expression>current.cmo_pid = new.cmo_pid and new.cmo_pid is not null</expression>
            </condition>
            <condition class="com.ataccama.nme.engine.model.lifecycle.ColumnsEqualCondition">
                <columns>
                    <column>cmo_first_name</column>
                    <column>cmo_last_name</column>
                </columns>
            </condition>
        </conditions>
    </lifeCycle>
</authoring>

Master Match Operation

This operation defines the way instances are matched to authored master records. For more information, see Mixed Style in MDM, section Master Matching.

Parameters:

  • <matchingColumns>: defines instance entity columns that need to be matched with master entity columns.

    • name: name of the source column.

    • expression: matching expression.

Lifecycle

The parameter defines changes that are made to the record based on set conditions.

Parameters:

  • <conditions>: condition that need to be met for entity to lose ownership. All conditions listed in this section must be fulfilled.

  • class: defines the type of the condition that needs to be met.

    • ExpressionCondition : condition based on an expression.

      • <expression>: specifies the expression using current and new dot source inputs.

    • ColumnsEqualCondition: condition based on equality of columns.

      • <column>: defines columns that need to be the same in current and new records for the condition to be met.

External Entities

Defines external entities and their behavior on the master layer.

<entities>
  <externalEntities>
    <externalEntity name="audit">
      <columns>
    ..
      </columns>
      <relationships>
    ..
      </relationships>

    </externalEntity>
  </externalEntities>

Column

Column definition for master entities is done this way:

<columns>
    <column name="event_type" type="string" size="50"/>
    <column name="event_timestamp" type="datetime"/>
    <column name="event_user" type="string" size="50"/>
</columns>

Parameters:

  • name: the name of the column

  • type: data type

  • size: column length (used as a database column size) that is mandatory for string data type

External Relationship

The external relationship specifies the source of the external data and the way the external entity is mapped to other entities.

<relationships>
  <rel name="insured_events" parentEntity="party" reverseName="insured_events">
    <dataProvider class="com.ataccama.nme.dqc.external.ExternalDataProviderConfig" planFileName="../engine/external/masters_insured_event/insured_events.comp">
      <columns>
        <column name="cmo_pid"/>
      </columns>
      <relationships/>
    </dataProvider>
  </rel>
</relationships>

nme-model.gen.xml Example

The following configuration does not show all configuration possibilities. 
<?xml version='1.0' encoding='UTF-8'?>
<model>
  <instanceLayer>
    <entities>
      <entity name="party">
        <columns>
          <column name="src_name" origin="source" type="string" size="100"/>
          <column name="std_name" origin="clean" type="string" size="100"/>
          <column name="exp_name" origin="clean" type="string" size="500"/>
          <column name="sco_name" origin="clean" type="integer" size="10"/>
          <column name="std_first_name" origin="clean" type="string" size="100"/>
          <column name="std_last_name" origin="clean" type="string" size="100"/>
          <column name="src_ssn" origin="source" type="string" size="100"/>
          <column name="std_ssn" origin="clean" type="string" size="100"/>
          <column name="exp_ssn" origin="clean" type="string" size="500"/>
          <column name="sco_ssn" origin="clean" type="integer" size="10"/>
          <column name="uni_can_id" origin="match" type="long" size="20"/>
          <column name="uni_rule_name" origin="match" type="string" size="100"/>
          <column name="uni_role" origin="match" type="string" size="10"/>
          <column name="add_std_zip" origin="copy_clean" type="string" size="1000">
              <valueSource columnName="std_zip" relationshipName="pat_add">
                     <aggregation maxLength="1000" class="com.ataccama.nme.engine.model.ConcatenateDistinct" separator="~"/>
              </valueSource>
          </column>
          <column name="agg_group_count" origin="aggregate" type="string" size="1000"/>
        </columns>
        <relationships/>
        <matchingOperation class="com.ataccama.nme.dqc.operations.MatchingPlan" planFileName="../trans/party/party_match.comp"/>
        <cleansingOperation class="com.ataccama.nme.dqc.operations.CleansingPlan" planFileName="../trans/party/party_clean.comp"/>
        <aggregatingOperation class="com.ataccama.nme.dqc.operations.AggregatingPlan" groupingColumn="master_id" planFileName="../engine/trans/party/party_aggregate.comp"/>
    </entity>
      <entity name="address">
                <columns>
                    <column name="pty_source_id" origin="source" type="string" size="100"/>
                    <column name="src_street" origin="source" type="string" size="100"/>
                    <column name="std_street" origin="clean" type="string" size="100"/>
                    <column name="src_city" origin="source" type="string" size="100"/>
                    <column name="std_city" origin="clean" type="string" size="100"/>
                    <column name="src_state" origin="source" type="string" size="100"/>
                    <column name="std_state" origin="clean" type="string" size="100"/>
                    <column name="src_zip" origin="source" type="string" size="100"/>
                    <column name="std_zip" origin="clean" type="string" size="100"/>
                    <column name="exp_address" origin="clean" type="string" size="500"/>
                    <column name="sco_address" origin="clean" type="integer" size="10"/>
                    <column name="pty_master_id" origin="copy_clean" type="long_int" size="30">
                        <valueSource columnName="master_id" relationshipName="party">
                            <aggregation class="com.ataccama.nme.engine.model.FirstValue"/>
                        </valueSource>
                    </column>
                </columns>
                <relationships>
                    <rel reverseName="addresses" parentEntity="party" parentColumn="source_id" name="party" foreignKeyColumn="pty_source_id">
                        <type class="com.ataccama.nme.engine.model.SameSystemRelationship"/>
                    </rel>
                </relationships>
                <matchingOperation class="com.ataccama.nme.dqc.operations.MatchingPlan" planFileName="../trans/address/address_match.comp"/>
                <cleansingOperation class="com.ataccama.nme.dqc.operations.CleansingPlan" planFileName="../trans/address/address_clean.comp"/>
            </entity>
        </entities>
    <sourceSystems>
            <system name="crm">
                <instanceMappings>
                    <originId name="crm_customer#Party" entity="party" code="10"/>
                    <originId name="crm_address#Address" entity="address" code="11"/>
                </instanceMappings>
            </system>
            <system name="life">
                <instanceMappings>
                    <originId name="life_contract#Party" entity="party" code="20"/>
                    <originId name="life_contract#Address" entity="address" code="21"/>
                </instanceMappings>
            </system>
        </sourceSystems>
    </instanceLayer>
    <masterLayers>
        <masterLayer name="Master">
            <entities>
                <entity name="party" class="com.ataccama.nme.engine.model.PersistentMasterEntity">
                    <columns>
                        <column name="cmo_first_name" type="string" size="100"/>
                        <column name="cmo_last_name" type="string" size="100"/>
                        <column name="cmo_id_card" type="string" size="30"/>
                        <column name="cmo_ssn" type="string" size="30"/>
                    </columns>
                    <relationships/>
                    <consolidation instanceEntity="party" groupingColumn="master_id">
                        <mergingOperation customActivityTracking="false" class="com.ataccama.nme.dqc.operations.MergingPlan" planFileName="../trans/party/Master_party_merge.comp"/>
                            <recordFilterExpression>eng_active=true</recordFilterExpression>
                        </mergingOperation>
                    </consolidation>
                    <authoring/>
                </entity>
                <entity name="address" class="com.ataccama.nme.engine.model.PersistentMasterEntity">
                    <columns>
                        <column name="pty_id" type="long_int" size="30"/>
                        <column name="cmo_street" type="string" size="100"/>
                        <column name="cmo_city" type="string" size="100"/>
                        <column name="cmo_state" type="string" size="100"/>
                        <column name="cmo_zip" type="string" size="100"/>
                    </columns>
                    <relationships>
                        <rel reverseName="addresses" parentEntity="party" parentColumn="id" name="party" foreignKeyColumn="pty_id"/>
                    </relationships>
                    <consolidation instanceEntity="address" groupingColumn="master_id">
                        <mergingOperation customActivityTracking="false" class="com.ataccama.nme.dqc.operations.MergingPlan" planFileName="../trans/party/Master_address_merge.comp"/>
                            <recordFilterExpression>eng_active=true</recordFilterExpression>
                        </mergingOperation>
                    </consolidation>
                    <authoring/>
                </entity>
                <entity name="party_instance" instanceEntity="party" class="com.ataccama.nme.engine.model.VirtualInstanceEntity" >
                    <relationships>
                        <rel parentEntity="party" parentColumn="id" name="fk_party_inst" foreignKeyColumn="master_id"/>
                    </relationships>
                </entity>
                <externalEntities>
                    <externalEntity name="audit_drafts">
                        <columns>
                            <column name="event_type" type="string" size="50"/>
                            <column name="event_timestamp" type="datetime"/>
                            <column name="event_user" type="string" size="50"/>
                            <column name="comment" type="string"/>
                        </columns>
                        <relationships>
                            <rel name="party_has_drafts" parentEntity="party" reverseName="party_has_drafts">
                                <dataProvider class="com.ataccama.nme.dqc.external.ExternalDataProviderConfig" planFileName="../engine/external/provider_audit_drafts/party_has_drafts.comp">
                                    <columns>
                                        <column name="id"/>
                                    </columns>
                                    <relationships/>
                                </dataProvider>
                            </rel>
                        </relationships>
                    </externalEntity>
            </entities>
        </masterLayer>
    </masterLayers>
    <eventHandlersConfigFile>nme-event_handler.gen.xml</eventHandlersConfigFile>
</model>

Was this page useful?