If you can't find the product or version you're looking for, visit support.ataccama.com/downloads

Orchestration Server in Ataccama Cloud

Every Ataccama Cloud environment is deployed with a default configuration of the Orchestration Server (also called ONE Runtime Server). If you don’t require any modifications to the default configuration, no action is needed.

However, the Orchestration Server lets you extend the functionality and define additional data source connections that would otherwise not be available.

The following article describes how to work with the Orchestration Server in Ataccama Cloud environments.

Limitations

The following workflow tasks are not available:

Run Shell Script, Run Windows Command: Cannot be used due to security reasons.
Run DQC on Cluster: Use Run DPM Job instead.
HTTP Get: Temporarily disabled.

Prerequisites

To make changes to the Orchestration Server configuration in an Ataccama Cloud environment, the following requirements must be in place:

You have an up and running environment in Ataccama Cloud.
You have a Git repository ready to store and deploy your server configuration from.

Add the orchestration template to your Git repository

The orchestration server template contains all files and the folder structure necessary to start developing additional configuration.

You can download the current configuration template from the following link: Orchestration Server configuration template.

Add the contents of the template to your Git repository. The folder structure of your repository should be as follows:

<your-repository>
- orch
  - Files
    
    notificationHandlers
    
    schedulers
    
    workflows
  - runtime

Clone your repository to ONE Desktop

To establish a connection to your Git repository and create a project that is also suitable for local testing, we recommend you clone your repository to ONE Desktop.

You can download the necessary version of ONE Desktop from our Download page.

To learn more about how to work with ONE Desktop, see Welcome to ONE Desktop.

To clone your repository:

Launch ONE Desktop.
Switch the perspective to Git.
1. Select to the Open Perspective icon.
2. Select Git.
3. Select Open. This opens the Git Repositories browser in the sidebar.
From the Git Repositories browser, select Clone a Git repository.
Fill out the necessary information to clone your repository using your preferred connection protocol. The following image is an example of a valid https connection.
Select Next.
Choose the branches to clone from your repository and then select Next.
Configure the local destination of your repository and then select Finish. This clones your repository to the specified local destination and adds it to the Git Repositories browser.
Right-click the repository and then select Import Projects.
Select Finish. This adds the repository to your DQ Projects.
To view the project, switch back to the ONE Perspective.

You now have a ONE Desktop project from which you can commit and push changes to your Git repository.

For more information about how to work with Git in ONE Desktop, search through local ONE Desktop documentation and refer to Git.

Test your orchestration server configuration locally

You can run your custom configuration locally to test whether it starts without issues. However, the filepaths necessary to make this work within ONE Desktop are different than those in the Cloud Portal (the configuration inteface of your Ataccama Cloud environment).

To test your configuration locally:

Move the server.runtimeConfig and server.serverConfig files located withing the runtime folder and place them directly within the top folder of your project, that is on the same level as the orch folder.

We recommend moving rather than copying configuration files. This way, you avoid creating two file versions, which could lead to the wrong version being uploaded to the server.
Configure your project and the moved configuration files as needed.
Right-click the server.serverConfig file and then select Run Server Configuration.
In the Console window you can see whether your configuration started successfully.

To view additional details, you can access the Admin Center of your configuration of the Orchestration Server at http://localhost:8888.
To end the local test, select the Terminate icon in the Console window.

After you test your configuration locally, move the server.runtimeConfig and server.serverConfig files back to the runtime folder and commit them to your Git repository.

Connect your Git repository to your cloud environment

To deploy your Orchestration Server configuration from your Git repository to your cloud environment:

Go to Ataccama Cloud Portal, that is, your configuration interface.
Select an environment from the list of environments.
Go to the Services tab.
In the Shared Services section, find the Orchestration server service and select Settings.
In Edit connection, provide the following information:
1. Connection: Choose between HTTPS and SSH connection type.
2. Git server: Provide the Git server URL.
3. Initial branch: Specify the initial branch to be used.
4. Provide authentication credentials depending on the chosen connection type.
  - For HTTPS connection provide the following:
    
    Git username.
    
    Git password.
  - For SSH connection upload a Private SSH key.
Select Apply Git configuration to complete the setup. This restarts your environment and updates the server configuration.

During the update, the environment status is Updating, which means you can’t use any application. Once the environment becomes available again, you can start using the Orchestration Server as well as other applications.

You can now make changes to the configuration and push these changes to your cloud environment from ONE Desktop.

The Git repository holding the server configuration is queried for updates every 60 seconds. If a change is detected, it is immediately applied and the Orchestration Server is restarted.

If there are any running jobs when this occurs, these jobs are stopped. To prevent this from happening, make sure to commit configuration changes after all jobs are completed.

The server configuration can later be verified and managed in the Admin Console.

To access the Admin Console, select the Orchestration Server from the Shared Services section on the Services tab of your environment. For more information, see ONE Runtime Server Admin.

In case there is an issue, for example, the server configuration can’t be loaded or it contains errors, Admin Center is not available and the server log is displayed instead. Check both the server configuration and the settings in Cloud Portal before repeating the steps provided here.

Orchestration server configuration

In the context of Ataccama Cloud, the Orchestration Server configuration is structured as follows:

runtime folder contains the following two files:
- server.runtimeConfig: Used to define runtime configuration.
- server.serverConfig: Used to define additional configuration of server components.
Files folder contains files and resources necessary to run certain components, such as workflows, schedulers, and notification handlers.

`server.runtimeConfig`

The server.runtimeConfig XML file is structured as follows:

<runtimeconfig>
    <dataSources/>
    <databaseDrivers/>
    <parallelismLevel>1</parallelismLevel>
</runtimeconfig>

You can use it to define data source connections. For a list of supported data sources, see the Supported data sources table.

Example of data source connection definition.

<runtimeconfig>
    <dataSources>
        <dataSource
                password="password"
                name="database-name"
                driverName="postgresql"
                user="user"
                url="jdbc:postgresql://<DB_URL>/postgres">
            <properties>
                <property name="connectTimeout" value="0" />
            </properties>
            <propertiesEncrypted/>
            <propertiesFile/>
            <propertiesFileEncrypted/>
        </dataSource>
    </dataSources>
    <databaseDrivers/>
    <parallelismLevel>1</parallelismLevel>
</runtimeconfig>

Additionally, the databaseDrivers property lets you define data source connection including database drivers. For more information about database drivers, see Database drivers.

The parallelismLevel property defines the initial number of threads with which a step is started. The default value is 1. We recommend not exceeding the number of CPUs in the system when defining this value, see Parallelism level.

`server.serverConfig`

The server.serverConfig XML file accepts the same definitions that you would use for a locally run Runtime Server. For information about specific components, see Server Components.

However, some limitations apply:

The Online Services component cannot be used with Ataccama Cloud.
HTTP Dispatcher component: Ataccama Cloud environments come with a single predefined HTTP listener. If you define additional HTTP listeners in server.serverConfig, their configuration is merged and applied to the predefined HTTP listener.

Files

The Files folder contains any other configuration plans and associated plans. The data is organized into folders, as defined in their respective configuration files in server.serverConfig. For example, Files can hold folders such as:

workflows: Workflow files and any associated plans. See Workflow and Scheduler.
schedulers: Scheduler files and any associated plans. See Workflow and Scheduler.
notificationHandlers: Notification handler files and any associated plans. See Notifications Handler.

Edit and configure these files as needed, depending on which services and resources you want to set up.

Supported data sources

The following table is a list of JDBC data sources currently supported in Ataccama Cloud environments. When specifying a new connection, the driver name (driverName) is required and must match the values provided in the following table.

You can only use the drivers that you selected when creating your environment in the Cloud Portal (Applications > Data Quality and Governance > Drivers). The following drivers are not available in the current version: Cloudera, Databricks, Derby Network Server, SAP RFC.

Data source Driver name Enabled by default Data source Driver name Enabled by default

Data source	Driver name	Enabled by default	Data source	Driver name	Enabled by default
PostgreSQL	`postgresql`	Yes	MySQL	`mysql`	No
Amazon Aurora PostgreSQL	`aurora-postgresql`	Yes	Amazon Aurora MySQL	`aurora-mysql`	No
Oracle	`oracle`	Yes	Informix	`informix`	No
MSSQL	`mssql`	Yes	SAP HANA	`sap-hana`	No
MSSQL (NOLOCK)	`mssql-no-locks`	Yes	Salesforce	`cdata-salesforce`	No
Azure Synapse Analytics	`azure-synapse`	Yes	Dremio	`dremio`	No
MariaDB	`mariadb`	Yes	Redis	`redis`	No
Teradata	`teradata`	Yes	SQLite	`sqlite`	No
MSSQL (jTDS)	`jtds`	Yes	Airtable	`airtable`	No
Sybase	`sybase`	Yes	Cosmos DB	`cosmosdb`	No
Amazon Redshift	`redshift`	Yes	Mongo DB	`mongodb`	No
Azure Data Explorer	`adx`	Yes	Sybase Native	`sybase-native`	No
Fabric Lakehouse SQL Endpoint	`fabric-lakehouse-sql-endpoint`	Yes	JSON CData	`jsonCdata`	No
Big Query	`bq`	Yes	Snowflake	`snowflake`	Yes
Derby	`derbyclient`	Yes

PostgreSQL

postgresql

Yes

MySQL

mysql

Amazon Aurora PostgreSQL

aurora-postgresql

Yes

Amazon Aurora MySQL

aurora-mysql

Oracle

oracle

Yes

Informix

informix

MSSQL

mssql

Yes

SAP HANA

sap-hana

MSSQL (NOLOCK)

mssql-no-locks

Yes

Salesforce

cdata-salesforce

Azure Synapse Analytics

azure-synapse

Yes

Dremio

dremio

MariaDB

mariadb

Yes

Redis

redis

Teradata

teradata

Yes

SQLite

sqlite

MSSQL (jTDS)

jtds

Yes

Airtable

airtable

Sybase

sybase

Yes

Cosmos DB

cosmosdb

Amazon Redshift

redshift

Yes

Mongo DB

mongodb

Azure Data Explorer

adx

Yes

Sybase Native

sybase-native

Fabric Lakehouse SQL Endpoint

fabric-lakehouse-sql-endpoint

Yes

JSON CData

jsonCdata

Big Query

bq

Yes

Snowflake

snowflake

Yes

Derby

derbyclient

Yes

Was this page useful?