Loading Additional Drivers

This article describes how to add external drivers not included with the MDM packages.

Overview

Drivers for some of the third-party data sources and other components are not included in standard packages due to the size or licensing reasons. It is, however, possible for the MDM Server to load those additional drivers from a designated external drivers folder named lib-ext. The individual JARs can be stored in subfolders.

See the following sections for examples of configuration and lists of drivers required by the third-party components.

Data sources

HBase

Drivers
Required JAR
`cdata.jdbc.apachehbase.jar`

The required configuration is as follows:

mdm.runtimeConfig

<dataSource driverClass="cdata.jdbc.apachehbase.ApacheHBaseDriver" url="jdbc:apachehbase:Server=127.0.0.1;Port=8080;" name="HBase-CData" user="<user_name>" password="<password>">
    <properties/>
    <propertiesEncrypted/>
    <propertiesFile/>
    <propertiesFileEncrypted/>
</dataSource>

Message queue providers

Amazon SQS

Drivers
Required JARs
`cif.jms.sqs-<VERSION>.jar` `avatica-core-1.11.0.jar`

The required configuration is as follows:

mdm.serverConfig

<component class="com.ataccama.dqc.jms.JmsProviderComponent">
    <connectionPoolSize>5</connectionPoolSize>
    <jmsResources>
        <resource>awssqsmdm</resource>
    </jmsResources>
</component>

mdm.runtimeConfig

<config class="com.ataccama.dqc.jms.config.JmsContributor">
    <jmsConnections>
        <jmsConnection connectionFactory="QueueConnectionFactory" name="awssqsmdm">
            <contextParams>
                <contextParam name="java.naming.factory.initial" value="com.ataccama.dqc.jms.sqs.SQSInitialContextFactory"/>
                <contextParam name="java.naming.provider.url" value="https://sqs.eu-central-1.amazonaws.com/773634404733/awssqsmdmqueue"/>
                <contextParam name="region" value="eu-central-1"/>
                <contextParam name="queue.awssqsmdmqueue" value="awssqsmdmqueue"/>
                <contextParam name="authType" value="AWS_ACCESS_KEY"/>
                <contextParam name="accessKey" value="crypted:AES:{value}"/>
                <contextParam name="secretKey" value="crypted:AES:{value}"/>
            </contextParams>
        </jmsConnection>
    </jmsConnections>
</config>

Rabbit MQ, IBM Websphere MQ, Tibco MQ

Active MQ is included in the standard package.

Drivers
Rabbit MQ
`com.rabbitmq:amqp-client.jar`
IBM WebSphere
`com.ibm.mq.headers.jar` `com.ibm.mq.jar` `com.ibm.mq.jmqi.jar` `com.ibm.mqjms.jar`
Tibco MQ
`jms-2.0.jar` `slf4j-api-1.4.2.jar` `slf4j-simple-1.4.2.jar` `tibcrypt.jar` `tibjms.jar` `tibjmsadmin.jar` `tibjmsapps.jar` `tibjmsufo.jar` `tibrvjms.jar`

The required configuration is as follows:

mdm.serverConfig

<component class="com.ataccama.dqc.jms.JmsProviderComponent">
    <connectionPoolSize>5</connectionPoolSize>
    <jmsResources>
        <resource>{QUEUE NAME}</resource>
    </jmsResources>
</component>

mdm.runtimeConfig

<config class="com.ataccama.dqc.jms.config.JmsContributor">
    <jmsConnections>
        <jmsConnection connectionFactory="QueueConnectionFactory" name="{QUEUE NAME}">
            <contextParams>
                <contextParam name="java.naming.factory.initial"
                              value="org.apache.activemq.jndi.ActiveMQInitialContextFactory"/>
                <contextParam name="java.naming.provider.url" value="tcp://localhost:61616"/>
            </contextParams>
        </jmsConnection>
        ...
    </jmsConnections>
</config>

Other

Azure Data Lake Storage Gen 2

For configuration details, see Runtime Configuration > Azure Data Lake Storage 2.

mdm.runtimeConfig

<contributedConfigs>
    <config class="com.ataccama.dqc.azure.config.AzureGen2Contributor">
        <azureGen2Connections>
            <azureGen2Connection
                clientId="clientID"
                authenticateUser="false"
                <!-- authType="AAD_CLIENT_CREDENTIAL" -->
                <!-- authType="AAD_MANAGED_IDENTITY" -->
                clientKey="crypted:AES:encryptedKey"
                containerName="containerName"
                name="AzurGen2"
                storageAccount="storageAccount"
                authTokenEndpoint="https://login.microsoftonline.com/<tokenID>/oauth2/token" <!-- for use with AAD Service Principal-->
                clientId="clientID"/>
        </azureGen2Connections>
    </config>
</contributedConfigs>

Parquet File Reader

Drivers
Required JARs
`commons-configuration2-2.1.1.jar` `hadoop-auth-3.2.0.jar` `hadoop-common-3.2.0.jar` `hadoop-hdfs-client-3.2.0.jar` `hadoop-mapreduce-client-core-3.2.0.jar` `htrace-core4-4.1.0-incubating.jar` `parquet-hadoop-bundle-1.8.1.jar` `re2j-1.1.jar`
Additional JARs for use with AWS and Azure
`hadoop-aws-3.2.0.jar` `hadoop-azure-3.2.0.jar` `hadoop-azure-datalake-3.2.0.jar`

Salesforce

Drivers are included in the standard package.

The required configuration is as follows:

mdm.runtimeConfig

<config class="com.ataccama.extension.salesforce.dqc.config.SalesforceContributor">
      <salesforceConnections>
           <connection name="SFServer" credentialsType="login" password="<password>" secretToken="<secret_token>" username="<user_name>" />
      </salesforceConnections>
</config>

Was this page useful?