Lead your team forward
OCT 24 / 9AM ET Register nowSecurity and Audit
MDM has several security features:
-
auditing of events
-
authorization for executing operations and accessing data
-
data audit trail
nme-security.xml
The nme-security.xml file is a configuration of security.
It is usually stored in [MDC project]/etc/nme-security.xml
and referenced in the MDM Configuration file (see MDM Configuration).
Basic structure of nme-security.xml
is the following:
<?xml version='1.0' encoding='UTF-8'?>
<security>
<audit>
<writers>
<!-- one or more audit writers -->
</writers>
<eventFilters>
<!-- one or more event filters -->
</eventFilters>
</audit>
<authorization class="...">
<!-- configuration of authorization - permissions -->
</authorization>
</security>
Auditing
The engine can be configured to audit the following events:
-
execution of batch operations
-
calling a native service
-
reading a record via native services
-
writing (insert, update, delete) a record via native services
Configuration of an audit
consists of two parts:
-
Where the audit should be written to (
writers
element). -
What events should be audited (
eventFilters
element).
To enable auditing, add an audit
element to the configuration with at least one audit writer.
Audit events consist of several attributes and are written as JSON:
{"TYPE":"CALL_SERVICE","DATE":"2015-01-19 12:37:52","USER":"jsmith","ATTRIBUTES":{"NAME":"listpartyInstances","GROUP":"Read services"}}
A full log line generated by the server looks as follows.
16.01.2018 20:23:32 [INFO] [NmeServerComponent][NME-Auditing][WRITE_RECORD] [18] {"DATE":"2015-01-19 12:37:52","USER":"jsmith","TYPE":"CALL_SERVICE","ATTRIBUTES":{"NAME":"listpartyInstances","GROUP":"Read services"}}
where
-
[NmeServerComponent]
- the parent log trace indicating it comes from Server Component. -
[NME-Auditing]
- the logger as configured in the writer(s) -
[WRITE_RECORD]
- event type -
[18]
- MDM server task ID -
{JSON}
- event details
Writers
ServerLoggerAuditWriter
This writer writes audit events using the Logging Configuration of the MDM server (see Logging Configuration). You can set the name of the logger.
<writer class="com.ataccama.nme.engine.security.audit.ServerLoggerAuditWriter">
<loggerName>NME Auditing</loggerName> <!-- default name is "Audit" -->
</writer>
LoggerAuditWriter
This writer writes audit events using standalone configuration and a logging engine not connected to logging of the MDM server. You must provide the name of the configuration file. Structure of the configuration file is the same as for the online server Logging Component.
<writer class="com.ataccama.nme.engine.security.audit.LoggerAuditWriter">
<loggerConfig>nme-audit-logging.xml</loggerConfig>
</writer>
Event Filters
Event filters configure what events are audited.
Every event filter adds auditing of some kind of events.
You can omit eventFilters
element or configure no event filters, in that case AuditDefault
event filter will be used.
In high traffic environments, auditing every event may consume enormous amount of disk space. For example, one call to a native service providing 100 records leads to 101 audited events. Average size of an audited event is 200 bytes. In an environment with 100k calls a day, every day audit logs will take approximately 2GB of disk space. |
Example configuration of event filters:
...
<eventFilters>
<eventFilter class="com.ataccama.nme.engine.security.audit.filters.BatchOpsAll" />
<eventFilter class="com.ataccama.nme.engine.security.audit.filters.ServicesByName">
<services>
<service>identifyParty</service>
</services>
</eventFilter>
</eventFilters>
...
The table below lists all available event filters.
Event Filter Class | Description | Additional Configuration |
---|---|---|
AuditAll |
All events are audited. |
|
AuditDefault |
Default event filter if an audit is enabled but no event filers are configured. |
|
BatchOpsAll |
All executions of batch operations (load, export) are audited at the moment of request for execution is received. |
|
BatchOpsByName |
Executions of configured batch operations (load, export) are audited at the moment when a request for execution is received. |
A list of batch operation names. Names are evaluated as regular expressions. |
ServicesAll |
All calls of native services are audited. One call is one audit event. |
|
ServicesByName |
Calls of configured native services are audited. One call is one audit event. |
A list of native service names. Names are evaluated as regular expressions. |
ReadAll |
All reads of data via native services (getById, list, search, traversal, identify) are audited. Read of one record is one audit event. |
|
ReadInstanceLayer |
All reads of instance records are audited. Read of one record is one audit event. |
Optionally, a list of source systems. |
ReadMasterLayer |
All reads of master records are audited. Read of one record is one audit event. |
Optionally, a list of master views. |
ReadEntities |
All reads of records in configured entities are audited. Read of one record is one audit event. |
A list of entities. |
WriteAll |
All writes via native services are audited. One record on input is one audit event. |
|
WriteEntities |
Writes of records in configured entities are audited. One record on input is one audit event. |
A list of instance entities. |
Several event filters have additional parameters or specific behavior. Those are explained below.
AuditDefault
This default event filter audits all executions of batch operations and calls to native services belonging to the Special services or Write services groups.
BatchOpsByName
You have to provide a list of regular expressions that are evaluated against all configured batch operations. Execution of all matched batch operations is audited. Sample configuration:
<eventFilter class="com.ataccama.nme.engine.security.audit.filters.BatchOpsByName">
<batchOperations>
<op>.+_load</op>
<op>.*reprocess.*</op>
<op>crm_export</op>
</batchOperations>
</eventFilter>
ServicesByName
You have to provide a list of regular expressions that are evaluated against all configured batch services. A call of any matched native service is audited. Sample configuration:
<eventFilter class="com.ataccama.nme.engine.security.audit.filters.ServicesByName">
<services>
<service>get.+ById</service>
<service>invoke.*</service>
</services>
</eventFilter>
ReadInstanceLayer
You may provide a list of source systems, so that only reads of instance records from those systems are audited. Otherwise all reads of instance records are audited. Sample configuration:
<eventFilter class="com.ataccama.nme.engine.security.audit.filters.ReadInstanceLayer">
<sourceSystems>
<system>crm</system>
</sourceSystems>
</eventFilter>
ReadMasterLayer
You may provide a list of master views, so that only reads of master records from those views are audited. Otherwise all reads of master records are audited. Sample configuration:
<eventFilter class="com.ataccama.nme.engine.security.audit.filters.ReadMasterLayer">
<masterViews>
<view>invest</view>
</masterViews>
</eventFilter>
Audited Events Overview
The table below lists details of all possible audit events. Every audit event contains its type, username, and timestamp.
Event Type/Code | Description | Additional Attributes |
---|---|---|
RUN_BATCH |
A batch operation was requested. |
operation type, operation typeid, operation name, source of operation (workflow, service, command line, web console), source additional attributes (name of workflow, id of workflow instance, name of service, remote address) |
CALL_SERVICE |
A native service was called. |
service name, service group |
READ_RECORD |
A record was read via native services. |
record id, record sourceId (instance records only), record type, record layer, record masterView (master records only) |
WRITE_RECORD |
A request to write record was sent. If no action is taken (foe example, delete of a nonexisting record), an event is audited anyway. |
record id, record sourceId (instance records only), record type, record source system, record origin, record change type |
UNAUTHORIZED |
An unauthorized attempt was done. |
unauthorized event |
READ_TYPE |
A request to read some records was sent via native services. Used only as an unauthorized event inside an event of type UNAUTHORIZED. |
record type |
WRITE_TYPE |
A request to write some records of instance type was sent via native services. Used only as an unauthorized event inside event of type UNAUTHORIZED. |
instance record type |
Additional Attributes
Some additional audit parameters can be configured.
<audit>
...
<additionalAttributes>
<a class="">
...
</a>
</additionalAttributes>
...
</audit>
Including HTTP Headers in the Audit Events
To include HTTP headers in the audit events, add the following configuration. List all the headers that should be included into the audit event.
<audit>
...
<additionalAttributes>
<a class="com.ataccama.nme.engine.security.audit.HttpHeaders">
<headers>
<header name="Host" /> <!-- name of header -->
<header name="custom" />
</headers>
</a>
</additionalAttributes>
</audit>
The ATTRIBUTES
part of the audit event JSON output will be enhanced with the HTTP_HEADERS
:
{
...
"ATTRIBUTES":{
...
"HTTP_HEADERS":{
"Host":"localhost:8888",
"custom":"customvalue"
}
}
}
Audit Event Examples
Run batch operation
{
"TYPE":"RUN_BATCH",
"DATE":"2015-01-19 15:14:33",
"USER":"jsmith",
"ATTRIBUTES":{
"TYPE":"BATCH_LOAD",
"TYPEID":"batchLoad.crm_full",
"NAME":"Batch load \"crm_full\"",
"TASK_SOURCE":"service_call",
"TASK_SOURCE_ATTRIBUTES":{
"SERVICE_NAME":"invokeLoadOperation"
}
}
}
Call service
{
"USER":"jsmith",
"TYPE":"CALL_SERVICE",
"DATE":"2015-01-19 12:37:52",
"ATTRIBUTES":{
"NAME":"listpartyInstances",
"GROUP":"Read services"
}
}
Read record
{
"TYPE":"READ_RECORD",
"DATE":"2015-01-19 12:37:52",
"USER":"jsmith",
"ATTRIBUTES":{
"LAYER":"instance",
"ENTITY":"party",
"ID":"1218"
}
}
Write record
{
"TYPE":"WRITE_RECORD",
"DATE":"2015-01-19 15:20:12",
"USER":"jsmith",
"ATTRIBUTES":{
"ID":"1303209",
"SOURCE_ID":"1",
"ENTITY":"party",
"SOURCE_SYSTEM":"crm",
"ORIGIN":"crm#customer#party",
"CHANGE_TYPE":"NONE"
}
}
Authorization
You can configure the engine to require authorization of the following actions:
-
execution of batch operations
-
calling a native service
-
reading a record via native services
-
writing (insert, update, delete) a record via native services
Authentication (getting user identity and their roles and attributes) is configured in the MDM Server application.properties
file (see MDM Server Application Properties).
You can enable authorization by adding the authorization
element to the security configuration.
Currently there is only one type of authorization in role based evaluation - for which you have to specify a list of roles with assigned permissions:
<?xml version='1.0' encoding='UTF-8'?>
<security>
...
<authorization class="com.ataccama.nme.engine.security.auth.RoleBasedAuthorization">
<!-- Optional configuration, default is false -->
<authorizeDataServices>true</authorizeDataServices>
<roles>
<role name="esb">
<permissions>
<permission class="com.ataccama.nme.engine.security.auth.permissions.ServicesAll" />
...
</permissions>
</role>
</roles>
</authorization>
</security>
RoleBasedAuthorization
Because record reads and writes are done by native services, the user would need permission to access data and call native services. For example, to read a party record, a user needs permission to read the party and to call the listParty native service. To avoid this double permission requirement, by default calls to data services are not authorized (getbyId, list, search, traversal, identify).
If you want to force authorization of calls to data services, add authorizeDataServices
and set it to true as seen in the code block above.
Permissions
By combining several permissions you can create roles.
Table below lists all available permissions:
Permission class | Description | Additional configuration |
---|---|---|
AllPermissions |
Grants all permissions. |
|
BatchOpsAll |
Grants permission to execute all batch operations (load, export). |
|
BatchOpsByName |
Grants permission to execute configured batch operations. |
A list of batch operation names. Names are evaluated as regular expressions. |
ServicesAll |
Grants permission to call all native services. |
|
ServicesByName |
Grants permission to call configured native services. |
A list of native service names. Names are evaluated as regular expressions. |
ReadAll |
Grants permission to read all records. |
|
ReadInstanceLayer |
Grants permission to read all instance records. |
Optional, a list of source systems. |
ReadMasterLayer |
Grants permission to read all master records. |
Optional, a list of master views. |
ReadEntities |
Grants permission to read records of configured entities. |
A list of entities. Entity can optionally contain list of columns and record conditions. |
WriteAll |
Grants permission to write any record. |
|
WriteEntities |
Grants permission to write records in configured entities. |
A list of instance entities. Entity can optionally contain list of source systems and record conditions. |
Several permissions have additional parameters. Those are explained below.
Read* permissions are used in calls of native services that read data, particularly getById, list, search, traversal and identify (only for filtering output; internal identification ignores permissions to get the correct result). Write* permissions are used in calls to write native services.
BatchOpsByName
You have to provide a list of regular expressions that are evaluated against all configured batch operations. Permission to execute all matched batch operations is granted. Sample configuration:
<permission class="com.ataccama.nme.engine.security.auth.permissions.BatchOpsByName">
<batchOperations>
<op>crm_load</op>
<op>.*_export</op>
</batchOperations>
</permission>
ServicesByName
You have to provide a list of regular expressions that are evaluated against all configured batch services. Permission to call all matched native services is granted. Sample configuration:
<permission class="com.ataccama.nme.engine.security.auth.permissions.ServicesByName">
<services>
<service>get.+ById</service>
<service>identify.*</service>
</services>
</permission>
ReadInstanceLayer
You may provide a list of source systems that the role has permission to read from; otherwise, the role is granted permission to read all instance records. Sample configuration:
<permission class="com.ataccama.nme.engine.security.auth.permissions.ReadInstanceLayer">
<sourceSystems>
<system>crm</system>
</sourceSystems>
</permission>
ReadMasterLayer
You may provide a list of master views, so that the role has permission to read master records only from those views; otherwise, the role is granted permission to read all master records. Sample configuration:
<permission class="com.ataccama.nme.engine.security.auth.permissions.ReadMasterLayer">
<masterViews>
<view>invest</view>
</masterViews>
</permission>
ReadEntities
You have to provide a list of entities:
<permission class="com.ataccama.nme.engine.security.auth.permissions.ReadEntities">
<entities>
<entity name="address" layer="instance" />
<entity name="party" layer="master" masterView="invest" />
</entities>
</permission>
Optionally, the entity
element may contain a list of `column`s and `condition`s.
If there is at least one column, only configured columns are visible to the caller of the service. If there are conditions specified, only records passing all conditions are visible to the caller of the service (that is, multiple conditions are combined by the AND operator). Condition’s column can be any column defined in the model configuration and several internal engine columns (eng_system for instance entities and eng_active for all entities).
Access attempt to invisible columns and records is silent; however, access attempts to records of fully invisible entities (the user cannot view the whole entity) is considered unauthorized and is logged according to the configuration.
...
<entity name="address" layer="instance">
<columns>
<!-- only columns listed below are visible -->
<column>src_city</column>
<column>src_state</column>
</columns>
<conditions>
<condition column="std_type" value="HOME" />
<!-- more conditions, combined by logical AND -->
</conditions>
</entity>
...
WriteEntities
You have to provide a list of instance entities:
<permission class="com.ataccama.nme.engine.security.auth.permissions.WriteEntities">
<entities>
<entity name="address" />
<entity name="party" />
</entities>
</permission>
Optionally, the entity
element may contain a list of conditions
and sourceSystems
.
If conditions are defined, only the records passing all conditions are visible to the caller of the service (that is, multiple conditions are combined by the AND operator).
The caller of the service can write records to any of the source systems specified (that is, sourceSystems
are combined by the OR operator).
An attempt to write a record not passing the configured conditions and allowed source systems is considered an unauthorized attempt and is logged according to the configuration.
...
<entity name="address">
<conditions>
<condition column="std_type" value="HOME" />
<!-- more conditions, combined by logical AND -->
</conditions>
<sourceSystems>
<system>crm</system>
<!-- more source systems, combined by logical OR -->
</sourceSystems>
</entity>
...
Data Audit Trail
In some cases you might want to identify the user who initiated the MDM Consolidation process inside transition plans (cleanse, match, merge) or identify the user who initiated a batch export inside an export plan. The feature is not available in load plans now.
You can access it by using the getParameterValue()
function in a DQC expression.
Return type | Expression | Description |
---|---|---|
STRING |
getParameterValue("security.user") |
Returns user name |
STRING |
getParameterValue("security.user.<attribute_name>") |
Returns the value of attribute <attribute_name> |
BOOLEAN |
getParameterValue("security.role.<role_name>") |
Returns whether the authenticated user has role <role_name> |
Was this page useful?