Lead your team forward
OCT 24 / 9AM ET Register nowAudit API
The Audit module is an optional ONE component that provides APIs for audit logs collected from Metadata Management Module (MMM), and Data Processing Module (DPM). Audit logs are stored in a separate PostgreSQL database and can be queried through GraphQL.
The following information is tracked:
-
Operations: This includes all user-triggered actions, such as listing, updating, and deleting assets.
-
Assets: It is possible to log listed entities, that is, the data that users requested.
As MMM typically generates a large amount of data, the Audit module can be disabled if necessary. Alternatively, you can also set up filters that reduce how much information is retained.
Before you start
By default, the following MMD entities are audited: source
, location
, connection
, credential
, catalogItem
, attribute
.
To enable auditing on a specific entity, add the following MMD trait to the entity configuration: audit:auditEnabled
.
For more information, see Traits.
When it comes to entity operations, the Audit module tracks all CRUD (create, read, update, and delete) operations as well as a number of custom operations for each entity.
To retrieve a list of custom operations, query the ONE API using the following request.
Replace the entity name as needed, for example, entityName: "credential"
.
List custom entity operations for catalog items
query listCustomNodeOperations {
_modelMetadata {
entities(entityName: "catalogItem", depth: 1) {
operations(ownOperationsOnly: true, excludeNamespaces: "core") {
name
}
}
}
}
The following list contains all custom entity operations that are available by default for each entity.
Default custom entity operations
{
"catalogItem": [
{
"name": "catalog:partitionsInfo"
},
{
"name": "dqeval:runCatalogItemDqEval"
},
{
"name": "profiling:bulkProfile"
},
{
"name": "dqEvalCheck:checkCatalogItemDqEval"
},
{
"name": "catalog:preview"
},
{
"name": "profiling:profile"
}
],
"credentials": [
{
"name": "datasource:testConnection"
}
],
"location": [],
"attribute": [
{
"name": "dqeval:runAttributeDqEval"
},
{
"name": "dqEvalCheck:checkAttributeDqEval"
}
],
"source": [
{
"name": "datasource:documentationFlow"
}
],
"connection": [
{
"name": "datasource:documentationFlow"
},
{
"name": "datasource:browse"
},
{
"name": "datasource:preview"
},
{
"name": "datasource:bulkTestConnection"
},
{
"name": "datasource:testConnection"
},
{
"name": "datasource:import"
}
]
}
If you are using the default configuration for on-premise deployment, the API can be reached at localhost:8071/graphql
.
The port number is specified through the server.port
property in the module configuration.
For more information about how to configure the Audit module, see [configuring-the-audit-module].
The purpose of the guide is to explain how you can query audit records through GraphQL and provide you with a number of basic usage examples. For a brief overview of some key concepts in GraphQL, see ONE API. For a complete guide on working with GraphQL, refer to the official GraphQL tutorials: Introduction to GraphQL. |
Overview of the schema
This section describes only the fields relevant to operations
and assets
object types.
Other object types in the Audit GraphQL schema are consistent with GraphQL recommendations and can be obtained by inspecting the GraphQL schema.
The Operation
type has the following fields defined:
Field name | Data type | Description | ||
---|---|---|---|---|
|
String |
The module that logged the audit record, for example, |
||
|
String |
The type of action performed, for example,
|
||
|
String |
The identifier that links the asset to all the operations related to that asset. |
||
|
Long |
The date and time of the action. Expressed in milliseconds starting from 01/01/1970. |
||
|
String |
The type of asset, for example:
|
||
|
String |
Identifies the module or the user responsible for the action. The identity is provided in the following format along with the information about the module and user roles:
|
||
|
String |
The type of operation, for example,
|
The following fields are available for the Asset
type:
Field name | Data type | Description | ||
---|---|---|---|---|
|
Array of strings |
Shows any unauthorized attempts to access data in MMM. In that case, MMM does not return any data to the user and an audit log is created instead. For example, this covers the cases in which access permissions are not granted to the user or the requested asset does not exist.
|
||
|
String |
If the asset is of type If the asset is of type See |
||
|
String |
The type of action that the user performed, for example,
|
||
|
String |
The identifier that links the asset to all the operations related to that asset. |
||
|
Long |
The date and time of the action. Expressed in milliseconds starting from 01/01/1970. |
||
|
String |
The identifier of the asset. |
||
|
Enum |
The type of the asset. Allowed values:
|
GraphQL operations
List assets
To retrieve a list of all accessed assets and their details, use the following listAssets
query.
This lets you obtain the following information for each asset: the type, name, and identifier of the entity, the actions performed and the time when they occurred, any violations concerning the data, and the correlation identifier.
-
Listing assets
-
Response body
query listAssets {
assets {
edges {
node {
correlationId
type
name
id
action
violation
time
}
}
}
}
The query is expected to return the following structure:
{
"data": {
"assets": {
"edges": [
{
"node": {
"correlationId": "2141d7",
"type": "ENTITY",
"name": "source",
"id": "1a6b74ff-726d-472c-85ac-f193c68dcfdd",
"action": "READ",
"violation": null,
"time": 1617977162453
}
},
{
"node": {
"correlationId": "2141d7",
"type": "ENTITY",
"name": "source",
"id": "3bc4a6c0-cd9f-4e6f-a1c4-d0d5de161eee",
"action": "READ",
"violation": null,
"time": 1617977162453
}
},
{
"node": {
"correlationId": "2141d7",
"type": "ENTITY",
"name": "source",
"id": "b62a3108-d107-4041-bfe6-d03672a31625",
"action": "READ",
"violation": null,
"time": 1617977162453
}
},
{
"node": {
"correlationId": "2141d7",
"type": "ENTITY",
"name": "source",
"id": "ff6c5afa-d187-4172-81e4-c697c13aeb08",
"action": "READ",
"violation": null,
"time": 1617977162453
}
},
{
"node": {
"correlationId": "2141d7",
"type": "ENTITY",
"name": "source",
"id": "8e1ee98b-0ae3-4579-ba3e-ff238cd2f076",
"action": "READ",
"violation": null,
"time": 1617977162453
}
},
{
"node": {
"correlationId": "2141d7",
"type": "ENTITY",
"name": "source",
"id": "e8357b1c-ef36-4627-97fd-cb69aa2b85e1",
"action": "READ",
"violation": null,
"time": 1617977162453
}
},
...
]
}
}
}
If you want to filter assets by a particular field, possible options are shown in the following example:
query filterAssets {
assets(
actions: ["DELETE"]
violation: [""]
#correlationId: "79b3ec"
type: ENTITY
asset: {
assetName: "source"
#assetId: "e8357b1c-ef36-4627-97fd-cb69aa2b85e1"
}
time: {
oldest: 1600000000000
newest: 1800000000000
}
)
{
edges {
node {
correlationId
id
...
}
}
}
}
To paginate results, follow this example. For more information about pagination in GraphQL, see Pagination. Pagination - example queryGraphQL query for listing assets - pagination
Pagination - example responseListing assets query response body - pagination
|
List operations
You can list all operations logged using the following query:
-
Listing operations
-
Response body
query listOperations{
operations {
edges {
node {
operation
action
time
correlationId
user
asset
module
}
}
}
}
The query response contains an overview of all operations, including the operation, action, and asset type, correlation identifier, the module and the user responsible for the action, as well as the timestamp when the action was performed (in milliseconds).
{
"data": {
"operations": {
"edges": [
{
"node": {
"operation": "catalogItem",
"action": "FINISH_SUCCESS",
"time": 1617712185832,
"correlationId": "bee2e2",
"user": "SimpleUserIdentity(id=716b5f1e-d566-4ec8-bcd8-27a7ff1f53e5), roles=[MDM_admin, DQIT_admin, admin, MMM_user, MDM_user, DQIT_user, MMM_admin, RDM_user, default, DPP_admin, RDM_admin, CS_admin, MMM_export, RDM, DQIT_supervisor])",
"asset": "catalogItem",
"module": "MMM"
}
},
{
"node": {
"operation": "catalogItem",
"action": "READ",
"time": 1617712185568,
"correlationId": "bee2e2",
"user": "SimpleUserIdentity(id=716b5f1e-d566-4ec8-bcd8-27a7ff1f53e5), roles=[MDM_admin, DQIT_admin, admin, MMM_user, MDM_user, DQIT_user, MMM_admin, RDM_user, default, DPP_admin, RDM_admin, CS_admin, MMM_export, RDM, DQIT_supervisor])",
"asset": "catalogItem",
"module": "MMM"
}
},
{
"node": {
"operation": "catalogItem",
"action": "FINISH_SUCCESS",
"time": 1617710914217,
"correlationId": "eb41f4",
"user": "SimpleUserIdentity(id=716b5f1e-d566-4ec8-bcd8-27a7ff1f53e5), roles=[MDM_admin, DQIT_admin, admin, MMM_user, MDM_user, DQIT_user, MMM_admin, RDM_user, default, DPP_admin, RDM_admin, CS_admin, MMM_export, RDM, DQIT_supervisor])",
"asset": "catalogItem",
"module": "MMM"
}
},
{
"node": {
"operation": "catalogItem",
"action": "READ",
"time": 1617710913926,
"correlationId": "eb41f4",
"user": "SimpleUserIdentity(id=716b5f1e-d566-4ec8-bcd8-27a7ff1f53e5), roles=[MDM_admin, DQIT_admin, admin, MMM_user, MDM_user, DQIT_user, MMM_admin, RDM_user, default, DPP_admin, RDM_admin, CS_admin, MMM_export, RDM, DQIT_supervisor])",
"asset": "catalogItem",
"module": "MMM"
}
},
{
"node": {
"operation": "catalogItem",
"action": "FINISH_SUCCESS",
"time": 1617710912188,
"correlationId": "5e57d4",
"user": "SimpleUserIdentity(id=716b5f1e-d566-4ec8-bcd8-27a7ff1f53e5), roles=[MDM_admin, DQIT_admin, admin, MMM_user, MDM_user, DQIT_user, MMM_admin, RDM_user, default, DPP_admin, RDM_admin, CS_admin, MMM_export, RDM, DQIT_supervisor])",
"asset": "catalogItem",
"module": "MMM"
}
},
...
]
}
}
}
Operations can be filtered using one or more of the following options:
query filterOperations {
operations(
#correlationId: "bee2e2"
operations: ["catalogItem"]
actions: ["READ"]
#user: "716b5f1e-d566-4ec8-bcd8-27a7ff1f53e5"
modules: ["MMM"]
asset: {
assetName: "catalogItem"
#assetId: "7c4993f7-bb8c-4885-a4e5-d3930d043f8e"
}
time: {
oldest: 1600000000000
newest: 1800000000000
}
) {
edges {
node {
operation
action
...
}
}
}
}
Was this page useful?