ONE Desktop ONE DQ&C ONE MDM ONE RDM ONE Runtime Server
latest (15.4.0) 15.3.0 15.2.0 15.1.0 14.5.x 13.9.x 13.8.x 13.7.x 13.6.x 13.5.0 13.4.x 13.3.x 13.2.x
User Community Service Desk Downloads
If you can't find the product or version you're looking for, visit support.ataccama.com/downloads

Lineage Configuration

This article provides information about how to enable and configure Lineage via two options:

  • With MANTA.

  • By configuring Lineage Database.

Configuring lineage with MANTA

Configuring MANTA in DPE

To configure MANTA, the following properties need to be added to Data Processing Engine (DPE) configuration. This can be done through the Configuration Service, by editing the DPE deployment configuration, or by modifying the dpe/etc/application.properties file.

Property Data type Description

plugin.manta-integration.enabled

Boolean

If set to true, MANTA integration is enabled.

plugin.manta-integration.ataccama.one.manta-integration.user

String

The username for MANTA.

Default value: admin.

plugin.manta-integration.ataccama.one.manta-integration.password

String

The password for MANTA.

Default value: admin.

plugin.manta-integration.ataccama.one.manta-integration.url-admin-gui

String

The URL where MANTA Admin interface is available.

Default value: localhost:8181/manta-admin-gui/.

plugin.manta-integration.ataccama.one.manta-integration.url-viewer-gui

String

This property enables the connection with MANTA. Base URL of links to MANTA lineage in the format http(s)://<host>:<port>/<application context>.

For example: manta.mycompany.com:8443/manta-dataflow-server (without "/" in the end).

plugin.manta-integration.ataccama.one.manta-integration.workflow

String

The name of the workflow based on which MANTA processing is performed. A custom workflow is uploaded on DPE startup.

Default value: com_ataccama_one_run_all_13_9.

plugin.manta-integration.ataccama.one.manta-integration.connection-config-file

String

Points to the MANTA configuration file.

Default value: mantaIntegration.json.

plugin.manta-integration.ataccama.one.manta-integration.max-number-of-retries

Number

Configures how many times a request for status is sent if there is an exception.

Default value: 5.

Enabling MANTA in other ONE modules

After you have set up MANTA in DPE, you need to enable the integration in the following modules:

  • Data Processing Module (DPM)

  • ONE Web Application

  • Metadata Management Module (MMM)

The property is added either to the deployment configuration of the module in the Configuration Service or to the module’s etc/application.properties configuration file.

Property Data type Description

plugin.manta-integration.enabled

Boolean

Enables MANTA integration for the given module.
External lineage relationships can be further customized through MMM. For more information, see MMM Configuration, section Relationships graph configuration.

Enabling MANTA connection from the Lineage tab

To enable a direct link from the Lineage tab to MANTA, you need to add the server URL to MMM configuration. To do so, use the following property:

Property Data type Description

ataccama.one.relationships.external.mantaServerHost

String

This property enables the connection with MANTA. The URL where the MANTA server is running. Used to construct direct links to MANTA from the Lineage tab.

Default value: localhost:8181/.

lineage configuration view detailed lineage

plugin.relationships-graph.ataccama.one.relationships.external.linkParams

String

Sets the default link parameters for constructing links to MANTA view from the Lineage tab.

Default value: depth=3&direction=BOTH&filter=0&filterEdges=false&level=TOP.

Configuring lineage via Lineage DB

The Ataccama lineage information can be internally stored in a separate relational database, called lineage. The database should contain the following three tables: catalog_items, attributes, and edge.

  1. catalog_items: Stores information about catalog items.

    Field Description

    id

    The unique identifier of the record in the table.

    type

    Type of the attribute (e.g., Table, View).

    qualified_name

    Name of the column with path.

    For example, qualified_name is <server_name/database_name/schema_name/catalog_item_name>, where server_name is optional, defined on the connection.

    parent_qualified_name

    Name of the origin parent with path.

    name

    Name of the attribute.

    description

    Description of the catalog item.

    manta_hyperlink

    URL of the Manta Viewer to display catalog item.

  2. attributes: Holds information about catalog item attributes.

    Field Description

    id

    The unique identifier of the record in the table.

    type

    Type of the attribute (e.g., Column).

    qualified_name

    Name of the column with path.

    For example, qualified_name is <server_name/database_name/schema_name/catalog_item_name/attribute_name>, where server_name is optional, defined on the connection.

    parent_qualified_name

    Name of the origin parent with path.

    name

    Name of the attribute.

    description

    Description of the attribute.

    manta_hyperlink

    URL of the Manta Viewer to display attribute.

  3. edge: Holds information about the relationship between attributes of different catalog items. The relation is defined on the attribute identifier.

    Field Description

    id

    The unique identifier of the record in the table.

    from_edge

    Name of the destination edge with path.

    to_edge

    Name of the source edge with path.

To import custom lineage from an external source, these three tables should be populated accordingly. By default, the lineage is populated through loading the information about nodes and edges into the tables. After the initial loading, if any updates using the API are required, an online service can be created by configuring Ataccama ONE Desktop. The online service can be set up based on customer needs.

Was this page useful?