If you can't find the product or version you're looking for, visit support.ataccama.com/downloads

Audit API

The Audit module is an optional ONE component that provides APIs for audit logs collected from Metadata Management Module (MMM), Data Processing Module (DPM), and ONE Data (Data Management Module, or DMM). Audit logs are stored in a separate PostgreSQL database and can be queried through GraphQL.

The following information is tracked:

Operations: This includes all user-triggered actions, such as listing, updating, and deleting assets.
Assets: It is possible to log listed entities, that is, the data that users requested.

As MMM typically generates a large amount of data, the Audit module can be disabled if necessary. Alternatively, you can also set up filters that reduce how much information is retained.

Before you start

By default, the following MMD entities are audited: source, location, connection, credential, catalogItem, attribute, dmmCatalogItem, record.

To enable auditing on a specific entity, add the following MMD trait to the entity configuration: audit:auditEnabled. For more information, see Traits.

When it comes to entity operations, the Audit module tracks all CRUD (create, read, update, and delete) operations as well as a number of custom operations for each entity. To retrieve a list of custom operations, query the ONE API using the following request. Replace the entity name as needed, for example, entityName: "credential".

List custom entity operations for catalog items

query listCustomNodeOperations {
    _modelMetadata {
        entities(entityName: "catalogItem", depth: 1) {
            operations(ownOperationsOnly: true, excludeNamespaces: "core") {
            name
            }
        }
    }
}

The following list contains all custom entity operations that are available by default for each entity.

Default custom entity operations

{
    "catalogItem": [
        {
            "name": "catalog:partitionsInfo"
        },
        {
            "name": "transactionData:runTransactionDataAnalysis"
        },
        {
            "name": "dqeval:runCatalogItemDqEval"
        },
        {
            "name": "profiling:bulkProfile"
        },
        {
            "name": "dqEvalCheck:checkCatalogItemDqEval"
        },
        {
            "name": "anomalyDetection:attributeAnomalies"
        },
        {
            "name": "transactionData:runTransactionDataAnalysisWithConfig"
        },
        {
            "name": "catalog:preview"
        },
        {
            "name": "anomalyDetection:anomalies"
        },
        {
            "name": "profiling:profile"
        }
    ],
    "credentials": [
        {
            "name": "datasource:testConnection"
        }
    ],
    "location": [],
    "attribute": [
        {
            "name": "dqeval:runAttributeDqEval"
        },
        {
            "name": "anomalyDetection:anomalies"
        },
        {
            "name": "dqEvalCheck:checkAttributeDqEval"
        }
    ],
    "source": [
        {
            "name": "datasource:documentationFlow"
        }
    ],
    "connection": [
        {
            "name": "datasource:documentationFlow"
        },
        {
            "name": "datasource:browse"
        },
        {
            "name": "datasource:preview"
        },
        {
            "name": "datasource:bulkTestConnection"
        },
        {
            "name": "datasource:testConnection"
        },
        {
            "name": "datasource:import"
        }
    ]
}

If you are using the default configuration for on-premise deployment, the API can be reached at http://localhost:8071/graphql. The port number is specified through the server.port property in the module configuration. For more information about how to configure the Audit module, see [configuring-the-audit-module].

The purpose of the guide is to explain how you can query audit records through GraphQL and provide you with a number of basic usage examples. For a brief overview of some key concepts in GraphQL, see ONE API.

For a complete guide on working with GraphQL, refer to the official GraphQL tutorials: Introduction to GraphQL.

Overview of the schema

This section describes only the fields relevant to operations and assets object types. Other object types in the Audit GraphQL schema are consistent with GraphQL recommendations and can be obtained by inspecting the GraphQL schema.

The Operation type has the following fields defined:

Field name Data type Description

module

String

The module that logged the audit record, for example, MMM, DPM, or DMM.

action

String

The type of action performed, for example, OPERATION, FINISH_SUCCESS, READ.

For this field, the enumeration values are provided dynamically based on the input from the audited modules. To get a full list of values, query the following endpoint: operationActions.

violation

Array of strings

Shows any unauthorized attempts to access data in MMM. In that case, MMM does not return any data to the user and an audit log is created instead.

For example, this covers the cases in which access permissions are not granted to the user or the requested asset does not exist.

For this field, the enumeration values are provided dynamically based on the input from the audited modules. To get a full list of values, query the following endpoint: violations.

correlationId

String

The identifier that links the asset to all the operations related to that asset.

time

Long

The date and time of the action. Expressed in milliseconds starting from 01/01/1970.

assetName

String

The name of the asset.

assetId

String

The unique identifier of the asset.

assetType

String

The type of asset, for example: attribute, catalogItem, connection, credential, location, source.

For this field, the enumeration values are provided dynamically based on the input from the audited modules. To get a full list of values, query the following endpoint: assetNames.

userName

String

The name of the user.

userId

String

The unique identifier of the user.

user

String

Identifies the module or the user responsible for the action. The identity is provided in the following format along with the information about the module and user roles:

Module: ServiceIdentity(module=aicore, id=aicore-dev, roles=[MMM_admin]).
User: SimpleUserIdentity(id=716b5f1e-d566-4ec8-bcd8-27a7ff1f53e5), roles=[admin, MMM_user, MMM_admin, default, DPP_admin]).

operation

String

The type of operation, for example, catalogItem, catalogItems, connection, testConnection, source, sources, checkCatalogItemDqEval.

For this field, the enumeration values are provided dynamically based on the input from the audited modules. To get a full list of values, query the following endpoint: operationNames.

The following fields are available for the Asset type:

Field name Data type Description

violation

Array of strings

Shows any unauthorized attempts to access data in MMM. In that case, MMM does not return any data to the user and an audit log is created instead.

For example, this covers the cases in which access permissions are not granted to the user or the requested asset does not exist.

For this field, the enumeration values are provided dynamically based on the input from the audited modules. To get a full list of values, query the following endpoint: violations.

action

String

The type of action that the user performed, for example, READ, UPDATE, DELETE.

For this field, the enumeration values are provided dynamically based on the input from the audited modules. To get a full list of values, query the following endpoint: assetActions.

correlationId

String

The identifier that links the asset to all the operations related to that asset.

time

Long

The date and time of the action. Expressed in milliseconds starting from 01/01/1970.

assetName

String

The name of the asset.

assetId

String

The unique identifier of the asset.

type

Enum

The type of the asset.

Allowed values:

ENTITY: Refers to any metadata entity that MMM logged, that is, the asset that the user accessed.

For example, this can be a data source or a catalog item.
LINK: Only used for DPM audit records.

When an operation is logged in DPM, an asset of this type is created containing a link to the DPM Admin Console. Following the link displays all the available information about the action logged.

assetType

String

If the asset is of type ENTITY, this narrows down the entity type. Possible values: source, credential, catalogItem, dmmCatalogItem, record, attribute, location, connection.

If the asset is of type LINK, this contains a link to the DPM Admin Console.

See type.

GraphQL operations

List assets

To retrieve a list of all accessed assets and their details, use the following listAssets query.

This lets you obtain the following information for each asset: the type, name, and identifier of the entity, the actions performed and the time when they occurred, any violations concerning the data, and the correlation identifier.

Listing assets
Response body

GraphQL query for listing assets

query listAssets {
    assets {
        edges {
            node {
                correlationId
                assetId
                violation
                action
                assetName
                time
                type
                assetType
            }
        }
    }
}

The query is expected to return the following structure:

List assets query response body

{
    "data": {
        "assets": {
            "edges": [
                {
                    "node": {
                        "correlationId": "5fe609",
                        "assetId": "f0a239e4-ddeb-411c-9740-ab07c328272d",
                        "violation": null,
                        "action": "READ",
                        "assetName": "employee",
                        "time": 1640265566769,
                        "type": "ENTITY",
                        "assetType": "catalogItem"
                    }
                },
                {
                    "node": {
                        "correlationId": "553aef",
                        "assetId": "c69be342-b2f4-4d8f-bbd7-ec9bff3e08b8",
                        "violation": null,
                        "action": "READ",
                        "assetName": "sales",
                        "time": 1638878203949,
                        "type": "ENTITY",
                        "assetType": "location"
                    }
                },
                {
                    "node": {
                        "correlationId": "b95f46",
                        "assetId": "3bc4a6c0-cd9f-4e6f-a1c4-d0d5de161eee",
                        "violation": null,
                        "action": "READ",
                        "assetName": "AWS MySQL source",
                        "time": 1640265561490,
                        "type": "ENTITY",
                        "assetType": "source"
                    }
                },
                {
                    "node": {
                        "correlationId": "b95f46",
                        "assetId": "10dca55a-2be8-459b-a4e6-44dd0f018cfb",
                        "violation": null,
                        "action": "READ",
                        "assetName": "mysql aws",
                        "time": 1640265561437,
                        "type": "ENTITY",
                        "assetType": "connection"
                    }
                },
                ...
            ]
        }
    }
}

If you want to filter assets by a particular field, possible options are shown in the following example:

GraphQL query for listing assets - filtering

query filterAssets {
    assets(
        actions: ["DELETE"]
        violation: [""]
        #correlationId: "79b3ec"
        type: ENTITY
        asset: {
            assetName: "source"
            #assetId: "e8357b1c-ef36-4627-97fd-cb69aa2b85e1"
        }
        time: {
            oldest: 1600000000000
            newest: 1800000000000
        }
    )
    {
            edges {
                node {
                    correlationId
                    assetId
                    ...
            }
        }
    }
}

To paginate results, follow this example. For more information about pagination in GraphQL, see Pagination.

Pagination - example query

GraphQL query for listing assets - pagination

query listAssets {
    assets (
    skip:2
    size: 3
    #cursor: "cursorId"
    #order: ASC
    ) {
        edges {
            node {
                correlationId
                type
                assetName
                assetId
                action
                violation
                time
            }
            cursor
        }
        pageInfo {
            endCursor
            hasNext
        }
    }
}

Pagination - example response

Listing assets query response body - pagination

{
    "data": {
        "assets": {
            "edges": [
                {
                    "node": {
                        "correlationId": "5fe609",
                        "type": "ENTITY",
                        "assetName": "app",
                        "assetId": "aebc08e3-bc48-4c63-a1a0-81b34dde2d72",
                        "action": "READ",
                        "violation": null,
                        "time": 1640265566711
                    },
                    "cursor": "1640265566|711361000|d9bcfe24-cb9e-4afc-8f3c-5e9516eef095"
                },
                {
                    "node": {
                        "correlationId": "5fe609",
                        "type": "ENTITY",
                        "assetName": "car_owner",
                        "assetId": "8d061db1-2ac9-4424-8dc4-58ed099f31ff",
                        "action": "READ",
                        "violation": null,
                        "time": 1640265566683
                    },
                    "cursor": "1640265566|683406000|8b6ff74e-7953-45eb-957b-cb0263e20fc1"
                },
                {
                    "node": {
                        "correlationId": "5fe609",
                        "type": "ENTITY",
                        "assetName": "app_person",
                        "assetId": "5f0a70a6-35e7-4866-80bb-89dca2b08ac5",
                        "action": "READ",
                        "violation": null,
                        "time": 1640265566654
                    },
                    "cursor": "1640265566|654575000|a20d178a-7638-4111-b840-afac062afa2d"
                }
            ],
            "pageInfo": {
                "endCursor": "1640265566|654575000|a20d178a-7638-4111-b840-afac062afa2d",
                "hasNext": true
            }
        }
    }
}

List operations

You can list all operations logged using the following query:

Listing operations
Response body

GraphQL query for listing operations

query listOperations{
    operations {
        edges {
            node {
                assetId
                module
                action
                assetName
                correlationId
                time
                userName
                userId
                user
                operation
                assetType
                violation
            }
        }
    }
}

The query response contains an overview of all operations, including the operation, action, and asset type, correlation identifier, the module and the user responsible for the action, as well as the timestamp when the action was performed (in milliseconds).

Listing operations query response body

{
    "data": {
        "operations": {
            "edges": [
                {
                    "node": {
                        "operation": "anomalies",
                        "action": "FINISH_SUCCESS",
                        "violation": false,
                        "time": 1640265566789,
                        "correlationId": "5fe609",
                        "user": "SimpleUserIdentity(id=70f8e8e2-115c-4506-aba9-527c3c60437e), roles=[MMM_application-admin, MDM_admin, TEAM_GLOSSARY_ADMINS, DQIT_admin, admin, MMM_user, MDM_user, DQIT_user, AUDIT_admin, ONE_ADMIN, MMM_admin, RDM_user, MDA_superuser, default, DPP_admin, CS_admin, RDM_admin, MMM_export, RDM, ONE_PLATFORM_MONITORING, DQIT_supervisor, AUDITOR])",
                        "userId": "70f8e8e2-115c-4506-aba9-527c3c60437e",
                        "userName": "admin",
                        "assetType": "catalogItem",
                        "assetId": null,
                        "assetName": null,
                        "module": "MMM"
                    }
                },
                {
                    "node": {
                        "operation": "checkCatalogItemDqEval",
                        "action": "FINISH_SUCCESS",
                        "violation": false,
                        "time": 1639432416621,
                        "correlationId": "9983f0",
                        "user": "SimpleUserIdentity(id=70f8e8e2-115c-4506-aba9-527c3c60437e), roles=[MMM_application-admin, MDM_admin, TEAM_GLOSSARY_ADMINS, DQIT_admin, admin, MMM_user, MDM_user, DQIT_user, AUDIT_admin, ONE_ADMIN, MMM_admin, RDM_user, MDA_superuser, default, DPP_admin, CS_admin, RDM_admin, MMM_export, RDM, ONE_PLATFORM_MONITORING, DQIT_supervisor, AUDITOR])",
                        "userId": "70f8e8e2-115c-4506-aba9-527c3c60437e",
                        "userName": "admin",
                        "assetType": "catalogItem",
                        "assetId": null,
                        "assetName": null,
                        "module": "MMM"
                    }
                },
                {
                    "node": {
                        "operation": "DETAIL",
                        "action": "FINISH_SUCCESS",
                        "violation": false,
                        "time": 1639432416556,
                        "correlationId": "4e359c",
                        "user": "SimpleUserIdentity(id=70f8e8e2-115c-4506-aba9-527c3c60437e), roles=[MMM_application-admin, MDM_admin, TEAM_GLOSSARY_ADMINS, DQIT_admin, admin, MMM_user, MDM_user, DQIT_user, AUDIT_admin, ONE_ADMIN, MMM_admin, RDM_user, MDA_superuser, default, DPP_admin, CS_admin, RDM_admin, MMM_export, RDM, ONE_PLATFORM_MONITORING, DQIT_supervisor, AUDITOR])",
                        "userId": "70f8e8e2-115c-4506-aba9-527c3c60437e",
                        "userName": "admin",
                        "assetType": "catalogItem",
                        "assetId": "e662f5a9-3dcf-4705-93b7-155afeebade0",
                        "assetName": "vsalespersonsalesbyfiscalyearsdata",
                        "module": "MMM"
                    }
                },
                {
                    "node": {
                        "operation": "LIST",
                        "action": "READ",
                        "violation": false,
                        "time": 1638878203763,
                        "correlationId": "c19a9c",
                        "user": "SimpleUserIdentity(id=70f8e8e2-115c-4506-aba9-527c3c60437e), roles=[MMM_application-admin, MDM_admin, TEAM_GLOSSARY_ADMINS, DQIT_admin, admin, MMM_user, MDM_user, DQIT_user, AUDIT_admin, ONE_ADMIN, MMM_admin, RDM_user, MDA_superuser, default, DPP_admin, CS_admin, RDM_admin, MMM_export, RDM, ONE_PLATFORM_MONITORING, DQIT_supervisor, AUDITOR])",
                        "userId": "70f8e8e2-115c-4506-aba9-527c3c60437e",
                        "userName": "admin",
                        "assetType": "catalogItem",
                        "assetId": "0726c74e-fc9e-40ad-a29d-23ec1dac8769",
                        "assetName": "catalogItem",
                        "module": "MMM"
                    }
                },
                ...
            ]
        }
    }
}

Operations can be filtered using one or more of the following options:

GraphQL for listing operations - filtering

query filterOperations {
    operations(
        #correlationId: "bee2e2"
        operations: ["catalogItem"]
        actions: ["READ"]
        userIds: ["716b5f1e-d566-4ec8-bcd8-27a7ff1f53e5"]
        modules: ["MMM"]
        asset: {
            assetName: "catalogItem"
            #assetId: "7c4993f7-bb8c-4885-a4e5-d3930d043f8e"
        }
        time: {
            oldest: 1600000000000
            newest: 1800000000000
        }
    ) {
        edges {
            node {
                operation
                action
                ...
            }
        }
    }
}

Was this page useful?