ONE API
ONE API can be queried using GraphQL, a query language for working with APIs and a dedicated runtime system that fulfills the queries based on a type system that you defined specifically for your data.
This guide is intended to give you a brief overview of several key concepts and actions in GraphQL and equip you with a basic understanding of how data can be queried and manipulated through GraphQL in ONE. Before you proceed, take time to get familiar with how GraphQL functions using their official guide: Introduction to GraphQL.
|
Ataccama ONE comes with a preconfigured playground for trying out GraphQL queries against our platform.
The GraphQL Playground is available at the following URL: In addition to a graphical interface, the tool provides context assist when constructing queries and lets you preview queries as well as schema and type details. |
Work with GraphQL
Endpoints
All GraphQL queries are sent to a single endpoint where the GraphQL service is running.
Typically, that is localhost:8021/graphql.
The host corresponds to the location at which the HTTP server of Metadata Management Module (MMM) is available by default.
When working with subscriptions, queries are directed to localhost:8021/subscriptions.
Subscriptions are event-triggered operations that allow you to send updates to users in real time. After the client subscribes to a channel of events, events are pushed by the server without any further action required on the client’s side.
HTTP headers
Request type and Content-Type
GraphQL queries are sent as POST requests with the query itself provided in the request body in JSON format.
The request needs to include Content-Type (application/json) and Authorization (Basic or Bearer) headers.
Authentication
For testing purposes, you can use the Basic HTTP Authentication Scheme.
The authentication token for the Basic HTTP Authentication consists of a base64-encoded username and password combination.
For example, the username-password combination admin:admin is encoded to YWRtaW46YWRtaW4=.
For real-life applications, we recommend using more secure forms of authentication. For more information, see API Requests Authentication.
|
To include the |
Request body
GraphQL query example
In GraphQL, the query is specified as follows:
query q {
catalogItems(versionSelector: {draftVersion: true}) {
edges {
node {
gid
draftVersion {
name
}
}
}
}
}-
Root item: Each query begins with a root item starting from which nodes are retrieved. In this case, the root object is
catalogItems. -
Fields: Queries look for information contained in fields. Objects have a number of fields that can be used to filter query results.
-
Arguments: It is possible to narrow down the query results through arguments, which define additional features of the object.
In this case, the argument
draftVersionis supplied for the fieldversionSelector. Since the requested value of the argument istrue, the query is expected to return catalog items that are in draft or published state. -
Edges: A technical wrapper that allows you to fetch more information, such as
totalCount, and customize other fields related to pagination. This way, you can specify how many nodes the query returns. For more information, see [Pagination in GraphQL]. -
Node: Nodes are queried items. In this example, the query searches for two fields on each node,
gid, which is the unique identifier of the catalog item, anddraftVersion, for which thenameshould be retrieved.
The same query is then provided as query property in JSON format in the request body.
Additional fields include operationName and variables, which are optional.
However, operationName should be used when working with multiple operations simultaneously.
{
"operationName": "q",
"variables": {},
"query": "query q {catalogItems(versionSelector: {draftVersion: true}) {edges {node {gid draftVersion { name }}}}}"
}|
Make sure to escape any quotation marks used in the query property, for example: |
The query is expected to return a list of catalog items with their unique identifier and names.
Listing catalog items query response body - example
{
"data": {
"catalogItems": {
"edges": [
{
"node": {
"gid": "0726c74e-fc9e-40ad-a29d-23ec1dac8769",
"draftVersion": {
"name": "addresstype"
}
}
},
{
"node": {
"gid": "07c3923c-b726-4424-8ca4-b6bc7867a842",
"draftVersion": {
"name": "phonenumbertype"
}
}
},
{
"node": {
"gid": "0c93d74b-0aa1-40bf-a022-b6aaeb73b321",
"draftVersion": {
"name": "currencyrate"
}
}
},
{
"node": {
"gid": "11fbfa28-5b37-4e8b-b0f2-c3aa7d85c559",
"draftVersion": {
"name": "businessentityaddress"
}
}
},
{
"node": {
"gid": "15070fa8-08b5-4204-889e-acedf49b7bd1",
"draftVersion": {
"name": "salespersonquotahistory"
}
}
}
...
]
}
}
}Operation name and type
The operation name should be a meaningful name assigned to each request. If you are sending only one query at a time, using an operation name is not necessary. However, when making multiple queries at once, operation names are very helpful for interpreting results and debugging any issues.
A GraphQL operation can be one of the following types:
-
Query: Returns data without making any changes.
-
Mutation: Generally used when modifying data or starting processes.
-
Subscription: Refers to a real-time or continual consumption of events.
The operation type is declared at the beginning of the query.
query operationName {
...
}Variables
If you are using variables, these need to be declared after the operation name while the value is provided in the variables property in the request body.
In the following example, the gid variable is declared as a non-null globally unique identifier type (GID).
Values that cannot be empty are marked by an exclamation mark following the variable type (!).
In the query itself, the variable is passed as an argument using the syntax $<variable_name>.
In variables, you need to reference the variable name without any prefixes and supply its value.
{
"operationName": "q",
"variables": {
"gid": "0726c74e-fc9e-40ad-a29d-23ec1dac8769"
},
"query": "query q($gid: GID!) {catalogItem(gid: $gid) { ... }}"
}If you are using GraphQL Playground, variables are declared in the Query Variables section using the following syntax:
{
"gid": "0726c74e-fc9e-40ad-a29d-23ec1dac8769"
}Error handling
Requests sent to a GraphQL endpoint always return the HTTP 200 OK response.
Any errors are shown in the errors field in the returned JSON.
For example, if you attempt to publish an entity that no longer exists, the following response is returned:
{
"errors": [
{
"message": "The node was not found.",
"locations": [
{
"line": 2,
"column": 1
}
],
"path": [
"catalogItemPublish"
],
"extensions": {
"reason": "NOT_FOUND",
"code": "NOT_FOUND",
"gid": "07c3923c-b726-4424-8ca4-b6bc7867a842",
"time": {
"nano": 220422000,
"epochSecond": 1617648635
},
"type": "catalogItem",
"hash": "dd4a08d8bed5c1f2e16c12205f39efd1",
"classification": "DataFetchingException"
}
}
],
"data": {
"catalogItemPublish": null
}
}When there is an issue with authentication, the following response is expected.
You can find more details in error_description in the message field.
{
"errors": [
{
"message": "{\"error\":\"invalid_grant\",\"error_description\":\"Invalid user credentials\"}",
"locations": [
{
"line": 2,
"column": 1
}
],
"path": [
"catalogItem"
],
"extensions": {
"reason": "INVALID_TOKEN",
"code": "UNAUTHORIZED",
"time": {
"nano": 800461000,
"epochSecond": 1617648843
},
"hash": "76ebda2290ffe334d9e54e98159eedaa",
"classification": "DataFetchingException"
}
}
],
"data": {
"catalogItem": null
}
}GraphQL operations
List entities
You can retrieve an entity, including its content and the embedded attributes, by the entity identifier. The operation also lets you query deep structures such as embedded entities and references.
It is also possible to apply primitive ordering (orderBy: [{ property: "name", direction: ASC }]), pagination (skip, size, cursor), and filtering (filter for AQL, query for AQL that is preprocessed by AI, queryParts` for user-friendly filters which are processed by AI).
For more information about how to use cursor-based pagination, see GraphQL’s guide about Pagination.
To view all catalog items that you can access, use the following query:
query listCatalogItems {
catalogItems(versionSelector: {draftVersion: true}) {
edges {
node {
gid
draftVersion {
name
attributes {
edges {
node {
gid
draftVersion {
name
}
}
}
}
}
}
}
}
}The query is expected to return a response in the following format:
Listing catalog items query response body - example
{
"data": {
"catalogItems": {
"edges": [
{
"node": {
"gid": "1589346f-d5b5-4400-9a13-ad3f733c70ea",
"draftVersion": {
"name": "party_full",
"attributes": {
"edges": [
{
"node": {
"gid": "0fb027e1-e680-496f-91a6-f43fb1b1e4eb",
"draftVersion": {
"name": "src_email"
}
}
},
{
"node": {
"gid": "1594ffae-ac27-447f-9204-4e412d1aecc7",
"draftVersion": {
"name": "src_gender"
}
}
},
{
"node": {
"gid": "202e490b-af4a-411a-903b-e1e75e4e7c85",
"draftVersion": {
"name": "meta_last_update"
}
}
},
{
"node": {
"gid": "2ece8b69-df78-4a95-abf9-cdbea8ebb252",
"draftVersion": {
"name": "src_sin"
}
}
},
{
"node": {
"gid": "6718299d-e931-454a-b9ee-7bbf628c57e8",
"draftVersion": {
"name": "src_card"
}
}
},
{
"node": {
"gid": "6da73afb-8ff3-436c-9e69-e20eaaf527a5",
"draftVersion": {
"name": "src_address"
}
}
},
{
"node": {
"gid": "8b2b919f-8357-4377-943b-99ca6718b4f3",
"draftVersion": {
"name": "src_name"
}
}
},
{
"node": {
"gid": "93b9e3c2-1cfb-47c0-95a3-47e43236f238",
"draftVersion": {
"name": "src_birth_date"
}
}
},
{
"node": {
"gid": "f231b0ad-ccd0-4aa5-b2f0-728c01822367",
"draftVersion": {
"name": "src_primary_key"
}
}
}
]
}
}
}
},
{
...
}
]
}
}
}To retrieve all available glossary terms, run the following query:
query listGlossaryTerms {
terms(versionSelector: {draftVersion: true}) {
edges {
node {
gid
draftVersion {
name
}
}
}
}
}The query returns the identifier and name of each glossary term:
Listing glossary terms query response body
{
"data": {
"terms": {
"edges": [
{
"node": {
"gid": "033df1b9-ef3c-48c7-b40a-a8fc98249221",
"draftVersion": {
"name": "Gender"
}
}
},
{
"node": {
"gid": "08fcb90f-763b-40bf-ae7d-0601e7ec1651",
"draftVersion": {
"name": "Airports"
}
}
},
{
"node": {
"gid": "194e2db8-f939-45ae-8ebd-db7cf8c10bab",
"draftVersion": {
"name": "First name"
}
}
},
{
"node": {
"gid": "8495a8d2-65b7-487d-acad-61b901324b58",
"draftVersion": {
"name": "Country"
}
}
}
...
]
}
}
}To get a list of all attributes of a specific catalog item, use the following query.
The catalog item identifier (gid) is provided as a variable and its value is specified outside of the query (see Variables).
query listAttributes ($gid: GID!) {
catalogItem(gid: $gid) {
publishedVersion {
attributes {
edges {
node {
gid
draftVersion {
name
}
}
}
}
}
}
}The response structure is shown in the following example:
Listing catalog item attributes query response body
{
"data": {
"catalogItem": {
"publishedVersion": {
"attributes": {
"edges": [
{
"node": {
"gid": "75642c37-8e88-4241-9a50-4f801eb47ade",
"draftVersion": {
"name": "name"
}
}
},
{
"node": {
"gid": "77215050-8ce9-4b12-94d9-2bc9265c2c07",
"draftVersion": {
"name": "rowguid"
}
}
},
{
"node": {
"gid": "808fef47-5dc8-4e60-8e91-e914cf18dec7",
"draftVersion": {
"name": "modifieddate"
}
}
},
{
"node": {
"gid": "c271635a-e3b0-499e-9bc3-81b8ae28a480",
"draftVersion": {
"name": "addresstypeid"
}
}
}
]
}
}
}
}
}To list all data sources and their data source connections, run the following query:
query listDataSources {
sources(versionSelector: { draftVersion: true }) {
edges {
node {
gid
draftVersion {
name
connections {
edges {
node {
gid
draftVersion {
name
}
}
}
}
}
}
}
}
}The query returns the following structure:
Listing data sources query response body
{
"data": {
"sources": {
"edges": [
{
"node": {
"gid": "1a6b74ff-726d-472c-85ac-f193c68dcfdd",
"draftVersion": {
"name": "RDM",
"connections": {
"edges": [
{
"node": {
"gid": "c94f2e3f-5a5e-4a7c-af0d-9d1da2020968",
"draftVersion": {
"name": "rdmapp"
}
}
}
]
}
}
}
},
{
"node": {
"gid": "3bc4a6c0-cd9f-4e6f-a1c4-d0d5de161eee",
"draftVersion": {
"name": "AWS MySQL source",
"connections": {
"edges": [
{
"node": {
"gid": "10dca55a-2be8-459b-a4e6-44dd0f018cfb",
"draftVersion": {
"name": "mysql aws"
}
}
}
]
}
}
}
},
{
"node": {
"gid": "8e1ee98b-0ae3-4579-ba3e-ff238cd2f076",
"draftVersion": {
"name": "Local Postgres source",
"connections": {
"edges": [
{
"node": {
"gid": "c7c17fbf-8482-46c0-8177-4040b285eab9",
"draftVersion": {
"name": "postgres local"
}
}
}
]
}
}
}
},
{
"node": {
"gid": "a4baa2d9-a399-4a10-ad93-21d45158d1de",
"draftVersion": {
"name": "AWS S3",
"connections": {
"edges": [
{
"node": {
"gid": "b2325313-30a6-4057-9cdf-bc5aa5845165",
"draftVersion": {
"name": "AWS S3"
}
}
}
]
}
}
}
},
{
"node": {
"gid": "b62a3108-d107-4041-bfe6-d03672a31625",
"draftVersion": {
"name": "Local filesystem",
"connections": {
"edges": [
{
"node": {
"gid": "2ec920b8-35f1-4fcb-9756-e113be0e79a2",
"draftVersion": {
"name": "local filesystem"
}
}
}
]
}
}
}
},
{
"node": {
"gid": "d5c951ee-2080-4460-8eea-e126e239ee3a",
"draftVersion": {
"name": "AWS Postgres source",
"connections": {
"edges": [
{
"node": {
"gid": "ad2553e8-af9e-4271-b354-a993494a2078",
"draftVersion": {
"name": "postgres aws"
}
}
}
]
}
}
}
},
{
"node": {
"gid": "e8357b1c-ef36-4627-97fd-cb69aa2b85e1",
"draftVersion": {
"name": "Compose PostgresTestData",
"connections": {
"edges": [
{
"node": {
"gid": "2e42a730-3a6e-4c50-ace7-f92b633edd65",
"draftVersion": {
"name": "tcd compose"
}
}
}
]
}
}
}
}
]
}
}
}Profile catalog items
To start profiling catalog items, you first need to retrieve information about available profiling configurations. To list all published profiling configurations, use the following query:
query listProfilingConfigurations {
profilingConfigurations(versionSelector: { publishedVersion: true }) {
edges {
node {
gid
publishedVersion {
name
type
description
anomalyDetectionEnabled
samplingLimitCount
samplingLimitPercentage
}
}
}
}
}There are two default profiling configurations: FULL and SAMPLE.
If there are no user-defined profiling configurations, the listProfilingConfigurations query returns only these two options.
If you select the SAMPLE profiling configuration, profiling covers only a subset of records in the catalog item.
A profiling configuration needs to be published before it can applied during profiling.
To specify which profiling configuration you want to use, you need to provide the unique identifier of that configuration (gid).
The listProfilingConfigurations query also returns details about the configuration itself, such as name, type, description, sampling limits, and information about anomaly detection.
Listing profiling configurations query response body
{
"data": {
"profilingConfigurations": {
"edges": [
{
"node": {
"gid": "723fa8c2-9ea9-4119-b0a7-d49f1f6e4c45",
"publishedVersion": {
"name": "Full profiling",
"type": "FULL",
"description": "Full profiling of all the table rows",
"anomalyDetectionEnabled": true,
"samplingLimitCount": null,
"samplingLimitPercentage": null
}
}
},
{
"node": {
"gid": "bb3815bc-6b30-49b0-94c1-d19915750aa4",
"publishedVersion": {
"name": "Sample profiling",
"type": "SAMPLE",
"description": "Sample profiling of table",
"anomalyDetectionEnabled": false,
"samplingLimitCount": 10000,
"samplingLimitPercentage": 20
}
}
}
]
}
}
}The following query triggers profiling on a catalog item.
query profiling {
catalogItem(gid: "0726c74e-fc9e-40ad-a29d-23ec1dac8769") {
gid
profile(defaultCredential: true, configId: "723fa8c2-9ea9-4119-b0a7-d49f1f6e4c45") {
gid
}
}
}These arguments are required:
-
gidon thecatalogItemfield. Refers to the unique identifier of the catalog item that should be profiled. -
configIdon theprofilefield. Points to the identifier (gid) of the selected profiling configuration, which was obtained using thelistProfilingConfigurationsquery.
Profiling catalog items query response body
{
"data": {
"catalogItem": {
"gid": "0726c74e-fc9e-40ad-a29d-23ec1dac8769",
"profile": {
"gid": "1f6fe8e6-5b19-439d-ad2f-99b60beaefda"
}
}
}
}To check the job status after the profiling has started, run the following query:
query getJobStatus {
job(gid: "5b899c99-bee5-4a5f-93e1-85a4d4768f41") {
gid
draftVersion {
status
}
}
}The gid argument on the job field matches the gid field on the profile item that is found in the results returned by the profiling query.
Retrieving job status query response body
{
"data": {
"job": {
"gid": "1f6fe8e6-5b19-439d-ad2f-99b60beaefda",
"draftVersion": {
"status": "FINISHED"
}
}
}
}In case the profiling job has failed, you can view the error by repeating the getJobStatus query with the added error field in the draftVersion field.
Connect to a data source and import metadata
To connect to a data source, you first need to obtain the source connection details. You can get a list of all data sources by running the following query (for more information, refer to List entities):
query listDataSources {
sources(versionSelector: { draftVersion: true }) {
edges {
node {
gid
draftVersion {
name
connections {
edges {
node {
gid
draftVersion {
name
}
}
}
}
}
}
}
}
}In the response, locate the data source and the identifier of the connection with which you want to work (gid).
This identifier is then used in the next query to connect to the source.
Listing data sources query response body
{
"data": {
"sources": {
"edges": [
{
"node": {
"gid": "8e1ee98b-0ae3-4579-ba3e-ff238cd2f076",
"draftVersion": {
"name": "Local Postgres source",
"connections": {
"edges": [
{
"node": {
"gid": "c7c17fbf-8482-46c0-8177-4040b285eab9", # Data source connection identifier
"draftVersion": {
"name": "postgres local"
}
}
}
]
}
}
}
},
...
]
}
}
}The following query establishes a connection with the selected source and imports the metadata for one of the tables.
If path is omitted, the query initiates metadata import for the whole source.
query connectToASource {
connection(gid: "a69e6a13-7863-44c7-9c4e-49aad1d431b3") {
import(defaultCredential: true, path: ["sourceName/schemaName/tableName"]) {
gid
}
}
}The query returns the import job identifier.
Importing metadata query response body
{
"data": {
"connection": {
"import": {
"gid": "5801073d-d8c3-4315-8e06-b82e85fc1ee4"
}
}
}
}You can then profile the imported catalog item. For more information, see Profile catalog items.
Run DQ evaluation
You can run data quality evaluation on a catalog item, a specific column in a catalog item, or a glossary term. Depending on the entity type, you need to provide the corresponding identifier. For more information about how to retrieve the correct identifier for each type of entity, see List entities.
In this case, the operation type is mutation (see Operation name and type).
mutation catalogItemDQ {
catalogItemEvaluateDq(gid: <catalog_item_id>) {
gid
}
}mutation catalogItemAttributeDQ {
attributeEvaluateDq(gid: <catalog_item_attribute_id>) {
gid
}
}mutation glossaryTermDQ {
termEvaluateDq(gid: <glossary_term_identifier>) {
gid
}
}All three operations return the same response format with the identifier of the DQ evaluation job (gid).
The name of the first field under data matches the executed mutation; in the following example, DQ evaluation was performed on a catalog item.
Other values include attributeEvaluateDq for DQ evaluation on catalog item attributes and termEvaluateDq for DQ evaluation on glossary terms.
Running DQ evaluation on a catalog item operation response body
{
"data": {
"catalogItemEvaluateDq": {
"gid": "e80174ca-a512-40e0-8975-b1417a63b4a9"
}
}
}In all three cases, the operation is expected to return the job identifier. You can further customize what the operation returns as needed.
Share entities
Before sharing an entity with another user or a group of users, start by exploring with which users that entity has already been shared (assignedRoles) and with which capabilities an entity can be shared (assignableCapabilities).
query GetCapabilities_catalogItem($gid: GID!, $path: [NodePath!]) {
catalogItem(gid: $gid) {
gid
operationSets {
name
displayName
description
id
order
__typename
}
assignableCapabilities(for: $path) {
name
displayName
description
id
operationSets {
name
displayName
description
id
order
__typename
}
__typename
}
assignedIdentities(for: $path) {
grantingStrategy
capability {
name
displayName
description
id
operationSets {
name
displayName
description
id
order
__typename
}
__typename
}
source {
nodePath
gid
__typename
}
identity {
... on Role {
name
description
__typename
... on GroupRole {
name
numberOfUsers
users {
id
... on Person {
username
firstName
lastName
email
userId
__typename
}
... on Service {
module
instanceId
__typename
}
__typename
}
gid
__typename
}
}
... on Person {
firstName
lastName
username
userId
email
id
__typename
}
__typename
}
__typename
}
__typename
}
}The following example shows the expected result of the query:
Listing sharing capabilities and assigned roles query response body
{
"extensions": {
"__$r": "i*Mc/Q& |79M~%&Y5R+b%qrfB!enr2*82NSj!"
},
"data": {
"catalogItem": {
"gid": "5f463b19-0000-7000-0000-0000000530aa",
"operationSets": [
{
"name": "owner",
"displayName": "Owner",
"description": "People with this role have full permissions for the asset (can publish changes and manage access).",
"id": "owner",
"order": 0,
"__typename": "OperationSet"
},
{
"name": "editor",
"displayName": "Editor",
"description": "People with this role have view access and permissions to edit this asset and publish changes. They can access the asset in all workflow states. They cannot manage access.",
"id": "editor",
"order": 1,
"__typename": "OperationSet"
},
{
"name": "viewer",
"displayName": "Viewer",
"description": "People with this role have view permissions for this asset. Editing or running data processes is not allowed. They have no permissions for workflow transitions.",
"id": "viewer",
"order": 2,
"__typename": "OperationSet"
}
],
"assignableCapabilities": [
{
"name": "editor",
"displayName": "Editor Everywhere",
"description": "The user has Editor role on this asset and all child entities.",
"id": "editor@/sources/locations/catalogItems",
"operationSets": [
{
"name": "editor",
"displayName": "Editor",
"description": "People with this role have view access and permissions to edit this asset and publish changes. They can access the asset in all workflow states. They cannot manage access.",
"id": "editor",
"order": 1,
"__typename": "OperationSet"
}
],
"__typename": "Capability"
},
{
"name": "owner",
"displayName": "Owner Everywhere",
"description": "The user has Owner role on this asset and all child entities.",
"id": "owner@/sources/locations/catalogItems",
"operationSets": [
{
"name": "owner",
"displayName": "Owner",
"description": "People with this role have full permissions for the asset (can publish changes and manage access).",
"id": "owner",
"order": 0,
"__typename": "OperationSet"
}
],
"__typename": "Capability"
},
{
"name": "viewer",
"displayName": "Viewer Everywhere",
"description": "The user has Viewer role on this asset and all child entities.",
"id": "viewer@/sources/locations/catalogItems",
"operationSets": [
{
"name": "viewer",
"displayName": "Viewer",
"description": "People with this role have view permissions for this asset. Editing or running data processes is not allowed. They have no permissions for workflow transitions.",
"id": "viewer",
"order": 2,
"__typename": "OperationSet"
}
],
"__typename": "Capability"
}
],
"assignedIdentities": [
{
"grantingStrategy": "GRANT",
"capability": {
"name": "super-admin",
"displayName": "Super Admin",
"description": "People with this role have full access and edit permissions for the application configuration and all assets in the application.",
"id": "super-admin@/",
"operationSets": [
{
"name": "owner",
"displayName": "Owner",
"description": "People with this role have full permissions for the asset (can publish changes and manage access).",
"id": "owner",
"order": 0,
"__typename": "OperationSet"
},
{
"name": "editor",
"displayName": "Editor",
"description": "People with this role have view access and permissions to edit this asset and publish changes. They can access the asset in all workflow states. They cannot manage access.",
"id": "editor",
"order": 1,
"__typename": "OperationSet"
}
],
"__typename": "Capability"
},
"source": {
"nodePath": "/",
"gid": "00000000-0000-0000-0000-000000000001",
"__typename": "PerNodeWrapper"
},
"identity": {
"name": "MMM_admin",
"description": "Has access to everything and can do everything",
"__typename": "GroupRole",
"numberOfUsers": 3,
"users": [
{
"id": "USER:b5372fa1-2486-4f9b-8a1a-1ca7fd0e6650",
"username": "admin",
"firstName": "Admin",
"lastName": null,
"email": "admin@admin.com",
"userId": "b5372fa1-2486-4f9b-8a1a-1ca7fd0e6650",
"__typename": "Person"
},
{
"id": "USER:95461b7d-28f2-4e4e-bb32-8ed092dff47e",
"username": "full-access-user",
"firstName": "first-name",
"lastName": "last-name",
"email": "full-access-user@ataccama.com",
"userId": "95461b7d-28f2-4e4e-bb32-8ed092dff47e",
"__typename": "Person"
},
{
"id": "USER:2ed92d57-5f79-4d0e-8a7a-b3272c7e91e4",
"username": "jane.smith",
"firstName": "Jane",
"lastName": "Smith",
"email": "jane.smith@ataccama.com",
"userId": "2ed92d57-5f79-4d0e-8a7a-b3272c7e91e4",
"__typename": "Person"
}
],
"gid": "08bc4396-0000-7000-0000-000000055ddc"
},
"__typename": "IdentityAssignment"
},
{
"grantingStrategy": "GRANT",
"capability": {
"name": "importedCatalog",
"displayName": "Immutable Catalog",
"description": "User can modify catalog structure",
"id": "importedCatalog@/",
"operationSets": [
{
"name": "importedCatalog",
"displayName": "importedCatalog",
"description": null,
"id": "importedCatalog",
"order": 3,
"__typename": "OperationSet"
}
],
"__typename": "Capability"
},
"source": {
"nodePath": "/",
"gid": "00000000-0000-0000-0000-000000000001",
"__typename": "PerNodeWrapper"
},
"identity": {
"name": "MMM_admin",
"description": "Has access to everything and can do everything",
"__typename": "GroupRole",
"numberOfUsers": 3,
"users": [
{
"id": "USER:b5372fa1-2486-4f9b-8a1a-1ca7fd0e6650",
"username": "admin",
"firstName": "Admin",
"lastName": null,
"email": "admin@admin.com",
"userId": "b5372fa1-2486-4f9b-8a1a-1ca7fd0e6650",
"__typename": "Person"
},
{
"id": "USER:95461b7d-28f2-4e4e-bb32-8ed092dff47e",
"username": "full-access-user",
"firstName": "first-name",
"lastName": "last-name",
"email": "full-access-user@ataccama.com",
"userId": "95461b7d-28f2-4e4e-bb32-8ed092dff47e",
"__typename": "Person"
},
{
"id": "USER:2ed92d57-5f79-4d0e-8a7a-b3272c7e91e4",
"username": "jane.smith",
"firstName": "Jane",
"lastName": "Smith",
"email": "jane.smith@ataccama.com",
"userId": "2ed92d57-5f79-4d0e-8a7a-b3272c7e91e4",
"__typename": "Person"
}
],
"gid": "08bc4396-0000-7000-0000-000000055ddc"
},
"__typename": "IdentityAssignment"
},
{
"grantingStrategy": "GRANT",
"capability": {
"name": "super-viewer",
"displayName": "Super Viewer",
"description": "People with this role have full access but no edit permissions for the application configuration and all assets in the application.",
"id": "super-viewer@/",
"operationSets": [
{
"name": "viewer",
"displayName": "Viewer",
"description": "People with this role have view permissions for this asset. Editing or running data processes is not allowed. They have no permissions for workflow transitions.",
"id": "viewer",
"order": 2,
"__typename": "OperationSet"
}
],
"__typename": "Capability"
},
"source": {
"nodePath": "/",
"gid": "00000000-0000-0000-0000-000000000001",
"__typename": "PerNodeWrapper"
},
"identity": {
"name": "MMM_read-only",
"description": "Has access to everything but cannot do anything",
"__typename": "GroupRole",
"numberOfUsers": 1,
"users": [
{
"id": "USER:c03aa9fc-93c2-4abd-a8bf-05cbbd44d799",
"username": "access-request-user",
"firstName": "first-name",
"lastName": "last-name",
"email": "access-request-user@ataccama.com",
"userId": "c03aa9fc-93c2-4abd-a8bf-05cbbd44d799",
"__typename": "Person"
}
],
"gid": "08bc4396-0000-7000-0000-000000055de2"
},
"__typename": "IdentityAssignment"
},
{
"grantingStrategy": "GRANT",
"capability": {
"name": "catalog-admin",
"displayName": "Catalog Admin",
"description": "The user has full permissions to all locations and catalog items as well as their relationships.",
"id": "catalog-admin@/",
"operationSets": [
{
"name": "owner",
"displayName": "Owner",
"description": "People with this role have full permissions for the asset (can publish changes and manage access).",
"id": "owner",
"order": 0,
"__typename": "OperationSet"
}
],
"__typename": "Capability"
},
"source": {
"nodePath": "/",
"gid": "00000000-0000-0000-0000-000000000001",
"__typename": "PerNodeWrapper"
},
"identity": {
"name": "MMM_data-manager",
"description": "Works mainly with catalog and related entities (data sources, rule library, components, lookups)",
"__typename": "GroupRole",
"numberOfUsers": 0,
"users": [],
"gid": "08bc4396-0000-7000-0000-000000055dbe"
},
"__typename": "IdentityAssignment"
},
{
"grantingStrategy": "GRANT",
"capability": {
"name": "catalog-admin",
"displayName": "Catalog Admin",
"description": "The user has full permissions to all locations and catalog items as well as their relationships.",
"id": "catalog-admin@/",
"operationSets": [
{
"name": "owner",
"displayName": "Owner",
"description": "People with this role have full permissions for the asset (can publish changes and manage access).",
"id": "owner",
"order": 0,
"__typename": "OperationSet"
}
],
"__typename": "Capability"
},
"source": {
"nodePath": "/",
"gid": "00000000-0000-0000-0000-000000000001",
"__typename": "PerNodeWrapper"
},
"identity": {
"name": "MMM_metadata-manager",
"description": "Manages metadata",
"__typename": "GroupRole",
"numberOfUsers": 1,
"users": [
{
"id": "USER:306a5a7b-5b6e-4ed5-b5bc-0790973e1ba0",
"username": "john.taylor",
"firstName": "John",
"lastName": "Taylor",
"email": "john.taylor@ataccama.com",
"userId": "306a5a7b-5b6e-4ed5-b5bc-0790973e1ba0",
"__typename": "Person"
}
],
"gid": "08bc4396-0000-7000-0000-000000055e33"
},
"__typename": "IdentityAssignment"
},
{
"grantingStrategy": "GRANT",
"capability": {
"name": "catalog-viewer",
"displayName": "Catalog Viewer",
"description": "The user has view permissions to all locations and catalog items as well as their relationships.",
"id": "catalog-viewer@/",
"operationSets": [
{
"name": "viewer",
"displayName": "Viewer",
"description": "People with this role have view permissions for this asset. Editing or running data processes is not allowed. They have no permissions for workflow transitions.",
"id": "viewer",
"order": 2,
"__typename": "OperationSet"
}
],
"__typename": "Capability"
},
"source": {
"nodePath": "/",
"gid": "00000000-0000-0000-0000-000000000001",
"__typename": "PerNodeWrapper"
},
"identity": {
"name": "MMM_data-analyst",
"description": "Accesses catalog, glossary and rule library",
"__typename": "GroupRole",
"numberOfUsers": 2,
"users": [
{
"id": "USER:628fb293-4791-4684-a588-22133d006b4d",
"username": "paul.james",
"firstName": "Paul",
"lastName": "James",
"email": "paul.james@ataccama.com",
"userId": "628fb293-4791-4684-a588-22133d006b4d",
"__typename": "Person"
},
{
"id": "USER:e8cd7540-0931-4b8e-98ce-3fe135d92a67",
"username": "rachel.adams",
"firstName": "Rachel",
"lastName": "Adams",
"email": "rachel.adams@ataccama.com",
"userId": "e8cd7540-0931-4b8e-98ce-3fe135d92a67",
"__typename": "Person"
}
],
"gid": "08bc4396-0000-7000-0000-000000055e2e"
},
"__typename": "IdentityAssignment"
},
{
"grantingStrategy": "GRANT",
"capability": {
"name": "catalog-viewer",
"displayName": "Catalog Viewer",
"description": "The user has view permissions to all locations and catalog items as well as their relationships.",
"id": "catalog-viewer@/",
"operationSets": [
{
"name": "viewer",
"displayName": "Viewer",
"description": "People with this role have view permissions for this asset. Editing or running data processes is not allowed. They have no permissions for workflow transitions.",
"id": "viewer",
"order": 2,
"__typename": "OperationSet"
}
],
"__typename": "Capability"
},
"source": {
"nodePath": "/",
"gid": "00000000-0000-0000-0000-000000000001",
"__typename": "PerNodeWrapper"
},
"identity": {
"name": "MMM_dq-specialist",
"description": "Works with monitoring projects and related entities (rule library, components, lookups)",
"__typename": "GroupRole",
"numberOfUsers": 1,
"users": [
{
"id": "USER:07cc08a2-5ad3-44ec-a60e-41883dd58757",
"username": "mark.goodwill",
"firstName": "Mark",
"lastName": "Goodwill",
"email": "mark.goodwill@ataccama.com",
"userId": "07cc08a2-5ad3-44ec-a60e-41883dd58757",
"__typename": "Person"
}
],
"gid": "08bc4396-0000-7000-0000-000000055e31"
},
"__typename": "IdentityAssignment"
},
{
"grantingStrategy": "GRANT",
"capability": {
"name": "catalog-viewer",
"displayName": "Catalog Viewer",
"description": "The user has view permissions to all locations and catalog items as well as their relationships.",
"id": "catalog-viewer@/",
"operationSets": [
{
"name": "viewer",
"displayName": "Viewer",
"description": "People with this role have view permissions for this asset. Editing or running data processes is not allowed. They have no permissions for workflow transitions.",
"id": "viewer",
"order": 2,
"__typename": "OperationSet"
}
],
"__typename": "Capability"
},
"source": {
"nodePath": "/",
"gid": "00000000-0000-0000-0000-000000000001",
"__typename": "PerNodeWrapper"
},
"identity": {
"name": "MMM_data-governor",
"description": "Defines policies and regulations",
"__typename": "GroupRole",
"numberOfUsers": 1,
"users": [
{
"id": "USER:628fb293-4791-4684-a588-22133d006b4d",
"username": "paul.james",
"firstName": "Paul",
"lastName": "James",
"email": "paul.james@ataccama.com",
"userId": "628fb293-4791-4684-a588-22133d006b4d",
"__typename": "Person"
}
],
"gid": "08bc4396-0000-7000-0000-000000055de0"
},
"__typename": "IdentityAssignment"
}
],
"__typename": "catalogItemNodeWrapper"
}
}
}To assign a sharing role, use the operation shown in the example.
When unassigning a sharing role, use the operation catalogItemUnassignCapabilities; the operation structure remains the same.
The two operations can be applied to every mmd node.
When sharing a business term, the root field is termAssignCapabilities.
The capabilities argument in the following mutation should be set to one of id fields of assignableCapabilities that you obtained through the query listSharingRolesAndCapabilities.
The same query returns the available options for the roles argument, which takes the value of the name field on role in assignedRoles.
With the following mutation, you can assign capabilities on a source or catalog item.
In the following mutation example body we use catalogItemAssignCapabilities to assign capabilities to a person or role on a catalog item.
You can use sourceAssignCapabilities to assign capabilities on a source.
mutation catalogItemAssignCapabilities($q0_gid: GID!, $q0_capabilities: [String!]!, $q0_roles: [String!]!, $q0_users: [String!]!, $q0_grantingStrategy: GrantingStrategy!, $q0_path: [NodePath!]) {
q0_catalogItemAssignCapabilities: catalogItemAssignCapabilities(
gid: $q0_gid
capabilities: $q0_capabilities
roles: $q0_roles
users: $q0_users
grantingStrategy: $q0_grantingStrategy
) {
result {
gid
assignedIdentities(for: $q0_path) {
grantingStrategy
capability {
id
__typename
}
identity {
... on Role {
name
__typename
}
... on Person {
id
__typename
}
__typename
}
__typename
}
__typename
}
__typename
}
}The response contains the identifier of the shared entity:
Sharing assets operation response body
{
"data": {
"q0_catalogItemAssignCapabilities": {
"result": {
"gid": "5f463b19-0000-7000-0000-0000000530aa",
"assignedIdentities": [
{
"grantingStrategy": "GRANT",
"capability": {
"id": "owner@/sources/locations/catalogItems",
"__typename": "Capability"
},
"identity": {
"name": "admin",
"__typename": "GroupRole"
},
"__typename": "IdentityAssignment"
},
{
"grantingStrategy": "GRANT",
"capability": {
"id": "super-admin@/",
"__typename": "Capability"
},
"identity": {
"name": "MMM_admin",
"__typename": "GroupRole"
},
"__typename": "IdentityAssignment"
},
{
"grantingStrategy": "GRANT",
"capability": {
"id": "importedCatalog@/",
"__typename": "Capability"
},
"identity": {
"name": "MMM_admin",
"__typename": "GroupRole"
},
"__typename": "IdentityAssignment"
},
{
"grantingStrategy": "GRANT",
"capability": {
"id": "super-viewer@/",
"__typename": "Capability"
},
"identity": {
"name": "MMM_read-only",
"__typename": "GroupRole"
},
"__typename": "IdentityAssignment"
},
{
"grantingStrategy": "GRANT",
"capability": {
"id": "catalog-admin@/",
"__typename": "Capability"
},
"identity": {
"name": "MMM_data-manager",
"__typename": "GroupRole"
},
"__typename": "IdentityAssignment"
},
{
"grantingStrategy": "GRANT",
"capability": {
"id": "catalog-admin@/",
"__typename": "Capability"
},
"identity": {
"name": "MMM_metadata-manager",
"__typename": "GroupRole"
},
"__typename": "IdentityAssignment"
},
{
"grantingStrategy": "GRANT",
"capability": {
"id": "catalog-viewer@/",
"__typename": "Capability"
},
"identity": {
"name": "MMM_data-analyst",
"__typename": "GroupRole"
},
"__typename": "IdentityAssignment"
},
{
"grantingStrategy": "GRANT",
"capability": {
"id": "catalog-viewer@/",
"__typename": "Capability"
},
"identity": {
"name": "MMM_dq-specialist",
"__typename": "GroupRole"
},
"__typename": "IdentityAssignment"
},
{
"grantingStrategy": "GRANT",
"capability": {
"id": "catalog-viewer@/",
"__typename": "Capability"
},
"identity": {
"name": "MMM_data-governor",
"__typename": "GroupRole"
},
"__typename": "IdentityAssignment"
}
],
"__typename": "catalogItemNodeWrapper"
},
"__typename": "catalogItemMutationResult"
}
}
}Delete entities
When deleting an entity, similarly to other edit operations such as creating or updating an entity, you first need to create a delete draft and then publish the draft. The operation requires the identifier of the entity.
To delete a catalog item attribute, replace the root item catalogItemDelete with attributeDelete.
When deleting a glossary term, use termDelete instead.
mutation deleteEntity {
catalogItemDelete(gid: "0726c74e-fc9e-40ad-a29d-23ec1dac8769") {
success
}
}The query returns the following structure:
Deleting catalog items operation response body
{
"data": {
"catalogItemDelete": {
"success": true
}
}
}For more information about how to publish the delete draft, see Publish entities.
Publish entities
Any mmd node can be published using the mutation publishEntity.
Currently, this operation bypasses the publishing workflow, which means that the approval request is automatically resolved (this behavior is expected to change later).
When publishing attributes, the root item should be attributePublish.
For glossary terms, use termPublish.
If the field result is added as shown in the example, the operation returns the name of the published node.
|
Depending on the type of entity, the
The content assist feature of GraphQL Playground can help you customize what the |
mutation publishEntity {
catalogItemPublish(gid: "0726c74e-fc9e-40ad-a29d-23ec1dac8769") {
success
result {
publishedVersion {
name
}
}
}
}The response is structured as follows:
Publishing entities operation response body
{
"data": {
"catalogItemPublish": {
"success": true,
"result": {
"publishedVersion": {
"name": "customers.csv"
}
}
}
}
}When publishing a delete draft, the operation returns a response in the following format:
Publishing a delete draft operation response body
{
"data": {
"catalogItemDelete": {
"success": true,
"result": {
"publishedVersion": null
}
}
}
}Duplicate rules and terms
It is possible to duplicate the configuration of rules and terms. This is useful when, for example, similar rules need to be assigned to multiple lookup files. This operation only copies the configuration and none of the entity relationships, which means that the duplicated rule or term is not assigned anywhere.
In both cases, you need to specify the identifier (gid) of the term or rule which you would like to duplicate.
mutation copyTerm {
termCopy(gid: "<term_ID>") {
result {
gid
}
}
}mutation copyRule {
ruleCopy(gid: "<rule_ID>") {
result {
gid
}
}
}In these examples, both operations return the identifier of the copied entity, for example:
Duplicating a rule operation response body
{
"data": {
"ruleCopy": {
"result": {
"gid": "a0ceca7a-1653-4338-9054-8da4e45dcd7e"
}
}
}
}Glossary terms
The following examples cover use cases specific for glossary terms, such as exporting term types and obtaining term statistics.
List term types
To retrieve a list of all terms types, along with term identifiers, use the following query. As needed, the query can be expanded to include additional information about terms (see List entities).
query getTermTypes {
terms(versionSelector: { draftVersion: true }) {
edges {
node {
gid
type
}
}
}
}Listing term types query response body
{
"data": {
"terms": {
"edges": [
{
"node": {
"gid": "01503672-d0a0-492f-9ac6-b318fb9ee7d2",
"type": "keyPerformanceIndicator"
}
},
{
"node": {
"gid": "029d5ece-d9ef-4dd9-a1c2-8e74289874d2",
"type": "keyPerformanceIndicator"
}
},
{
"node": {
"gid": "03b3e732-cded-40fd-9b52-134c2cfbefe3",
"type": "businessTerm"
}
},
...
]
}
}
}Retrieve term occurrences
To get an overview of the entities to which a specific glossary term has been assigned, run the following query.
The term identifier (gid) can be obtained using the previous query for listing term types.
When the withCount argument is set to true, the total count also includes the occurrences on child entities.
For example, this means that terms assigned to attributes are also counted for the catalog items containing those attributes.
To calculate occurrences of only directly assigned terms (that is, to exclude catalog item counts), set the value to false.
query getTermOccurrences {
term(gid: "033df1b9-ef3c-48c7-b40a-a8fc98249221") {
statistics(withCount: true, relationships: []) {
usageStatistics {
nodePath
totalCount
}
}
}
}The query returns a list of available entities and the number of times the term has been assigned to each entity type.
Retrieving term occurrences query response body
{
"data": {
"term": {
"statistics": {
"usageStatistics": [
{
"nodePath": "/sources/workspace/catalogItems/attributes",
"totalCount": 0
},
{
"nodePath": "/sources/locations/catalogItems",
"totalCount": 5
},
{
"nodePath": "/sources/locations",
"totalCount": 4
},
{
"nodePath": "/monitoringProjects",
"totalCount": 1
},
{
"nodePath": "/sources/workspace/catalogItems",
"totalCount": 0
},
{
"nodePath": "/sources/locations/catalogItems/attributes",
"totalCount": 0
},
{
"nodePath": "/components",
"totalCount": 0
},
{
"nodePath": "/rules",
"totalCount": 0
},
{
"nodePath": "/sources",
"totalCount": 3
},
{
"nodePath": "/sources/workspace",
"totalCount": 0
}
]
}
}
}
}Term aggregation results
This query aggregationResult returns aggregated result of DQ for a term.
It lives under the term entity.
It takes the last results from all attributes where the term is assigned and aggregates them.
Parameters:
-
at- specifies what moment in history we are interested. All runs after this time are excluded from the aggregation.
Example query:
query {
term(gid: "410c6bac-0000-7000-0000-000000051ce3") {
publishedVersion {
aggregationResult(at: "2022-07-26T09:34:27.117946Z") {
ruleInstanceCount
attributeCount
catalogItemCount
totalCount
results {
id
count
name
}
}
}
}
}Term attribute aggregation results
This query, attributeAggregationResult, returns results of term aggregation for attributes.
It lives under the term entity.
There are a number of parameters that can be defined:
-
Selector. Processing selector specifies which runs should be included in the results. You can use these 3 approaches:-
Limit: returns last n runs. Use the format
{limit: X}. -
Time range: returns runs in specified time range. Unspecified to parameter means until present time. Use the format seen below.
timeRange: { from: "2021-01-16T00:00:00.000000Z", to: "2021-01-17T00:00:00.000000Z" # `to` parameter is optional } -
Processing ids: list of specific processing ids. Use the format seen below.
selector: { processingIds: [ "34174311-f4b0-4efe-9462-272a066e6117", "00e71092-a53b-47f4-ae50-792b777609e7" ] }
-
-
attributes- List of attribute ids.
Example query:
query {
term(gid: "410c6bac-0000-7000-0000-000000051ce3") {
publishedVersion {
attributeAggregationResult(
selector: { limit: 1 }
attributes: [
{ attributeId: "5c866cdf-0000-7000-0000-000000055412" },
{ attributeId: "5c866cdf-0000-7000-0000-000000055408" }
]
) {
attributeId
aggregations {
aggregationId
processings {
processingId
startedAt
ruleInstanceCount
attributeCount
totalCount
partitionInfo {
partitions {
dataType
name
value
}
partitionType
}
results {
count
name
id
}
}
}
}
}
}
}Monitoring projects
Monitoring project queries live under the monitoringProject entity.
This section provides examples illustrating some common use cases for monitoring projects that should help you when constructing your own queries. These include running a monitoring project and retrieving monitoring results with different levels of detail.
Run a monitoring project
To run a monitoring project, you need the monitoring project identifier (projectId).
All project identifiers can be retrieved through the following query:
query getProjects {
monitoringProjects(versionSelector: { draftVersion: true }) {
edges {
node {
gid
}
}
}
}Listing monitoring projects query response body
{
"data": {
"monitoringProjects": {
"edges": [
{
"node": {
"gid": "0f8f3b16-a9a4-491e-9384-eb302dda1fba"
}
},
{
"node": {
"gid": "19fce03c-e238-49c9-ab26-ddddc48c95e0"
}
},
{
"node": {
"gid": "22f888c1-2dcc-466e-9b0f-95e19470ae27"
}
},
...
]
}
}
}Using the following query, DQ is monitored only on the most recent partition of the catalog item.
If partition options are not provided, monitoring uses the default catalog item settings defined in the project, which means that it runs either on the whole catalog item (FULL) or its latest partition (LATEST).
mutation startMonitoring {
runMonitoringProject(projectId: "89b662a3-8ece-409f-b523-01aeba8240a2",
partitionSettings: [
{
catalogItemId: "7c4993f7-bb8c-4885-a4e5-d3930d043f8e",
partitionType: LATEST
}
]
) {
loadMonitoringProjectId
}
}The query returns the following:
Running a monitoring project operation response body
{
"data": {
"runMonitoringProject": {
"loadMonitoringProjectId": "89b662a3-8ece-409f-b523-01aeba8240a2"
}
}
}Monitoring projects results global retention
To define a global retention policy on monitoring project results:
-
First create a configuration draft, with the retention value and unit defined as required. The mutation should be as follows:
GrapghQL mutation for setting global results retentionmutation createGlobalConfiguration{ projectResultsRetentionConfigurationCreate( parentNode: "dqConfiguration", parentProperty: "projectResultsRetentionConfiguration", parentGid:"00000000-0000-0000-0000-000000000001", new: { enabled: true, value:10, unit: "RUNS", # RUNS or DAYS _type: "projectResultsRetentionConfiguration"} ){ success aliasMapping { gid alias } result { gid } } } -
The response contains the GID of the newly created configuration:
Setting global results retention operation response body{ "data": { "projectResultsRetentionConfigurationCreate": { "success": true, "aliasMapping": [], "result": { "gid": "b8a87c0e-827d-46c0-9fa6-765ab5b39950" # <- id of new configuration } } } } -
Use the GID from the response and publish the draft:
GrapghQL mutation for publishing monitoring project configurationmutation publishConfig { projectResultsRetentionConfigurationPublish(gid: "b8a87c0e-827d-46c0-9fa6-765ab5b39950"){ success } }
To remove the configuration:
-
Send a
deletemutation using the GID of the configuration.GrapghQL mutation for removing monitoring project configurationmutation delete { projectResultsRetentionConfigurationDelete(gid: "b8a87c0e-827d-46c0-9fa6-765ab5b39950"){ success result { gid } } } -
The response contains a GID for the
deleteconfiguration. -
Publish the
deleteconfiguration using the GID from the previous response.GrapghQL mutation for publishing the configuration removalmutation publishConfig { projectResultsRetentionConfigurationPublish(gid: "b8a87c0e-827d-46c0-9fa6-765ab5b39950"){ success } }
DQ results
This query (dqCheckResultOverTime) returns results of selected DQ checks.
DQ checks of the type validity also include the explanation for invalid results.
There are a number of parameters that can be defined:
-
Selector- Specifies which runs should be included in the results. You can use these three approaches:-
Limit: Returns last n runs. Use the format
{limit: X}.By default, there is a limit of 10. This means, for example, that if you define a time range within which more than 10 processings are available, only results for 10 processings are returned. Increase the limit in combination with other parameters, such as time range, when required (maximum possible limit value is 2147483647). -
Time range: Returns runs in specified time range. Unspecified
toparameter means until present time. Use the following format:timeRange: { from: "2021-01-16T00:00:00.000000Z", to: "2021-01-17T00:00:00.000000Z" # `to` parameter is optional } -
Processing IDs: List of specific processing IDs. Use the following format:
selector: { processingIds: [ "34174311-f4b0-4efe-9462-272a066e6117", "00e71092-a53b-47f4-ae50-792b777609e7" ] }
-
-
Filter- Specifies which rows should be included in results. Filter attributes must match filter configuration of run.In other words, if run was started with filters enabled on attributes A and B, filter should also include these attributes. If the attributes do not match any run, the run is missing from results.
filter: [ { attributeId: "d3c8da98-0dea-4523-903c-cbc4a3f28e8b" values: ["Fire", "Water", "Grass"] } { attributeId: "00e71092-a53b-47f4-ae50-792b777609e7" values: ["Blastoise", "Bulbasaur", "Charizard"]} ]Values field can be empty. That has the same meaning as listing all possible values. Never use value #ALL#with other values as it gives incorrect results. You can use it alone but the preferred way is to use an empty list instead. -
catalogItemId- ID of the catalog item. -
checkIds- List of DQ check IDs (gidattribute ofdqCheckentity).
query CheckResults {
monitoringProject(gid: "957a400f-cfb7-4c0b-91f1-2f4ec441d734"){
publishedVersion {
dqCheckResultOverTime(
selector: {
limit: 2
},
catalogItemId: "6b03e87f-ae53-4c41-a16e-3c4b7234063b",
checkIds: [
"7b24c572-93be-462e-8b54-e76772b00f6d"
],
filter: [
{
attributeId: "d3c8da98-0dea-4523-903c-cbc4a3f28e8b"
values: ["Fire", "Water", "Grass"]
}
{
attributeId: "00e71092-a53b-47f4-ae50-792b777609e7"
values: ["Blastoise", "Bulbasaur","Charizard"]}
]
)
{
processingId
startedAt
results {
dqCheckId
results {
id
count
name
}
explanations {
id
count
name
}
alerts {
id
name
}
}
}
}
}
}The following response structure is returned.
Details
{
"data": {
"monitoringProject": {
"publishedVersion": {
"dqCheckResultOverTime": [
{
"processingId": "c770437f-e59c-4203-aa37-8093b475a59e",
"startedAt": "2021-10-25T09:54:30.871811Z",
"results": [
{
"dqCheckId": "7b24c572-93be-462e-8b54-e76772b00f6d",
"results": [
{
"id": "67b7aed6-b54f-4be3-a987-95899f605029",
"count": 3,
"name": "Valid"
},
{
"id": "ec49b8ed-52b2-469c-8424-4866249755cb",
"count": 3,
"name": "Invalid"
}
],
"explanations": [
{
"id": "5a69e2d1-c74c-43a2-a5a2-2705cc2a760d",
"count": 3,
"name": "IS_EMPTY"
}
],
"alerts": [
{
"id": "5a69e2d1-c74c-43a2-a5a2-2705cc2aabcd",
"name": "WARNING"
}
]
}
]
},
{
"processingId": "9ba1d3c4-9e01-4960-8344-4948a1373bb8",
"startedAt": "2021-10-27T09:22:59.491476Z",
"results": [
{
"dqCheckId": "7b24c572-93be-462e-8b54-e76772b00f6d",
"results": [
{
"id": "67b7aed6-b54f-4be3-a987-95899f605029",
"count": 3,
"name": "Valid"
},
{
"id": "ec49b8ed-52b2-469c-8424-4866249755cb",
"count": 3,
"name": "Invalid"
}
],
"explanations": [
{
"id": "5a69e2d1-c74c-43a2-a5a2-2705cc2a760d",
"count": 3,
"name": "IS_EMPTY"
}
],
"alerts": []
}
]
}
]
}
}
}
}Aggregation results
This query (dqAggregationResultOverTime) returns results of selected aggregations.
There are a number of parameters that can be defined:
-
Selector- Specifies which runs should be included in the results. You can use these three approaches:-
Limit: Returns last n runs. Use the format
{limit: X}.By default, there is a limit of 10. This means, for example, that if you define a time range within which more than 10 processings are available, only results for 10 processings are returned. Increase the limit in combination with other parameters, such as time range, when required (maximum possible limit value is 2147483647). -
Time range: Returns runs in specified time range. Unspecified
toparameter means until present time. Use the following format:timeRange: { from: "2021-01-16T00:00:00.000000Z", to: "2021-01-17T00:00:00.000000Z" # `to` parameter is optional } -
Processing IDs: List of specific processing IDs. Use the following format:
selector: { processingIds: [ "34174311-f4b0-4efe-9462-272a066e6117", "00e71092-a53b-47f4-ae50-792b777609e7" ] }
-
-
Filter- Specifies which rows should be included in results. Filter attributes must match filter configuration of run.In other words, if run was started with filters enabled on attributes A and B, filter should also include these attributes. If the attributes do not match any run, the run is missing from results.
filter: [ { attributeId: "d3c8da98-0dea-4523-903c-cbc4a3f28e8b" values: ["Fire", "Water", "Grass"] } { attributeId: "00e71092-a53b-47f4-ae50-792b777609e7" values: ["Blastoise", "Bulbasaur", "Charizard"]} ]Values field can be empty. That has the same meaning as listing all possible values. Never use value #ALL#with other values as it gives incorrect results. You can use it alone but the preferred way is to use an empty list instead. -
catalogItemId- ID of the catalog item. -
aggregationIds- ID of aggregation. There are three possible values that can be used:-
Custom aggregation ID.
-
Catalog item ID (aggregated results of all existing checks on the catalog item).
-
Attribute ID (aggregated results of all existing checks on the attribute).
-
query FetchProjectAggregationResults {
monitoringProject(gid: "957a400f-cfb7-4c0b-91f1-2f4ec441d734"){
publishedVersion {
dqAggregationResultOverTime(
selector: {
limit: 1,
},
catalogItemId: "6b03e87f-ae53-4c41-a16e-3c4b7234063b",
aggregationIds: [
"d046700c-f668-4d11-86bb-8af529deecf5"
"6b03e87f-ae53-4c41-a16e-3c4b7234063b"
],
filter: [
{
attributeId: "d3c8da98-0dea-4523-903c-cbc4a3f28e8b"
values: ["Fire", "Water", "Grass"]
}
{
attributeId: "00e71092-a53b-47f4-ae50-792b777609e7"
values: ["Blastoise", "Bulbasaur","Charizard"]}
]
){
processingId
startedAt
results {
aggregationId
results {
id
count
name
}
}
}
}
}
}The following response structure is returned:
Details
{
"data": {
"monitoringProject": {
"publishedVersion": {
"dqAggregationResultOverTime": [
{
"processingId": "9ba1d3c4-9e01-4960-8344-4948a1373bb8",
"startedAt": "2021-10-27T09:22:59.491476Z",
"results": [
{
"aggregationId": "6b03e87f-ae53-4c41-a16e-3c4b7234063b",
"results": [
{
"id": "67b7aed6-b54f-4be3-a987-95899f605029",
"count": 4,
"name": "Valid"
},
{
"id": "ec49b8ed-52b2-469c-8424-4866249755cb",
"count": 5,
"name": "Invalid"
}
]
},
{
"aggregationId": "d046700c-f668-4d11-86bb-8af529deecf5",
"results": [
{
"id": "67b7aed6-b54f-4be3-a987-95899f605029",
"count": 4,
"name": "Valid"
},
{
"id": "ec49b8ed-52b2-469c-8424-4866249755cb",
"count": 5,
"name": "Invalid"
}
]
}
]
}
]
}
}
}
}Monitoring project results
This query (dqProjectResultsOverTime) gives you project validity results over time.
The results are aggregated over all catalog items in a monitoring project.
The following parameter needs defining:
-
Selector- Specifies which runs should be included in the results. You can use these three approaches:-
Limit: Returns last n runs. Use the format
{limit: X}.By default, there is a limit of 10. This means, for example, that if you define a time range within which more than 10 processings are available, only results for 10 processings are returned. Increase the limit in combination with other parameters, such as time range, when required (maximum possible limit value is 2147483647). -
Time range: Returns runs in specified time range. Unspecified
toparameter means until present time. Use the following format:timeRange: { from: "2021-01-16T00:00:00.000000Z", to: "2021-01-17T00:00:00.000000Z" # `to` parameter is optional } -
Processing IDs: List of specific processing IDs. Use the following format:
selector: { processingIds: [ "34174311-f4b0-4efe-9462-272a066e6117", "00e71092-a53b-47f4-ae50-792b777609e7" ] }
-
query {
monitoringProject(gid: "957a400f-cfb7-4c0b-91f1-2f4ec441d734"){
publishedVersion {
dqProjectResultOverTime(
selector: {
limit: 10
}
)
{
processingId
startedAt
results {
aggregationId
results {
id
count
name
}
}
}
}
}
}Catalog items
Catalog result queries live under the catalogItem or attribute entities.
Attribute aggregation results
This query (dqAttributeAggregationResultOverTime) returns results of certain DQ aggregations for attributes.
There are a number of parameters that can be defined:
-
Selector- Specifies which runs should be included in the results. You can use these three approaches:-
Limit: Returns last n runs. Use the format
{limit: X}.By default, there is a limit of 10. This means, for example, that if you define a time range within which more than 10 processings are available, only results for 10 processings are returned. Increase the limit in combination with other parameters, such as time range, when required (maximum possible limit value is 2147483647). -
Time range: Returns runs in specified time range. Unspecified
toparameter means until present time. Use the following format:timeRange: { from: "2021-01-16T00:00:00.000000Z", to: "2021-01-17T00:00:00.000000Z" # `to` parameter is optional } -
Processing IDs: List of specific processing IDs. Use the following format:
selector: { processingIds: [ "34174311-f4b0-4efe-9462-272a066e6117", "00e71092-a53b-47f4-ae50-792b777609e7" ] }
-
-
Aggregation IDs - Optional list of aggregation IDs.
query {
catalogItem(gid: "5c866cdf-0000-7000-0000-0000000553fb") {
publishedVersion {
dqAttributeAggregationResultOverTime(
selector: { limit: 1 }
attributes: [
{
attributeId: "5c866cdf-0000-7000-0000-000000055408",
aggregationIds: ["0a2a2a64-ff0f-4e05-94c1-3413f09a50e0"]
},
{
attributeId: "5c866cdf-0000-7000-0000-000000055406"
}
]
) {
attributeId
aggregations {
aggregationId
processings {
processingId
partitionInfo {
partitions {
dataType
name
value
}
partitionType
}
ruleInstanceCount
attributeCount
totalCount
startedAt
results {
count
name
id
}
}
}
}
}
}
}Catalog item aggregation results
This query (aggregationResultOverTime) returns results of certain DQ aggregations for catalog items.
There are a number of parameters that can be defined:
-
Selector- Specifies which runs should be included in the results. You can use these three approaches:-
Limit: Returns last n runs. Use the format
{limit: X}.By default, there is a limit of 10. This means, for example, that if you define a time range within which more than 10 processings are available, only results for 10 processings are returned. Increase the limit in combination with other parameters, such as time range, when required (maximum possible limit value is 2147483647). -
Time range: Returns runs in specified time range. Unspecified
toparameter means until present time. Use the following format:timeRange: { from: "2021-01-16T00:00:00.000000Z", to: "2021-01-17T00:00:00.000000Z" # `to` parameter is optional } -
Processing IDs: List of specific processing IDs. Use the following format:
selector: { processingIds: [ "34174311-f4b0-4efe-9462-272a066e6117", "00e71092-a53b-47f4-ae50-792b777609e7" ] }
-
-
Aggregation IDs - Optional list of aggregation IDs. They can be IDs for specific dimensions or for the overall quality.
query {
catalogItem(gid: "5c866cdf-0000-7000-0000-0000000553fb") {
publishedVersion {
aggregationResultOverTime(
selector: { limit: 2 }
aggregationIds: ["0a2a2a64-ff0f-4e05-94c1-3413f09a50e0"]
) {
aggregationId
processings {
processingId
startedAt
ruleInstanceCount
attributeCount
totalCount
partitionInfo {
partitions {
dataType
name
value
}
partitionType
}
results {
count
name
id
}
}
}
}
}
}Catalog item validity
This query (validityAggr) returns the latest validity result for a catalog item.
query {
catalogItem(gid: "5c866cdf-0000-7000-0000-0000000553fb") {
publishedVersion {
validityAggr {
publishedVersion {
validCount
invalidCount
validColor
invalidColor
overallQuality
}
}
}
}
}Catalog item processing
This query (dqProcessings) returns a list of catalog item processings.
There are a number of parameters that can be defined:
-
skip- Pagination, how many results should be skipped. -
size- Pagination, result limit.
query {
attribute(gid: "5c866cdf-0000-7000-0000-000000055408") {
publishedVersion {
dqProcessings(size: 3) {
processingId
startedAt
partitionInfo {
partitionType
partitions {
dataType
name
value
}
}
}
}
}
}Attribute validity
This query (validityAggr) returns the latest validity result for an attribute.
query {
attribute(gid: "77215050-8ce9-4b12-94d9-2bc9265c2c07"){
publishedVersion {
validityAggr {
publishedVersion {
validCount
invalidCount
validColor
invalidColor
overallQuality
}
}
}
}
}Change rule reference strategy
By default, componentInstances has the trait core:reference with value versionStrategy = FIX_HCN.
This means the reference is fixed to the history change number at time of creation, and component rules might not use the latest component implementation.
To remove the trait and always reference the latest version instead, use the mutation:
mutation {
ruleComponentInstanceUpdateReferences(
gid: "4671cd3c-0000-7000-0000-000000060036",
referenceNodePaths: ["target"],
fixLatestHcn:true
) {
success
}
}| This mutation is generic and can be used to change traits on other entities within monitoring projects but needs to be executed separately for each entity. |
Was this page useful?