If you can't find the product or version you're looking for, visit support.ataccama.com/downloads

Configuring Tasks for Exporting Data to Text Files

The following steps explain how to configure a task for exporting data from RDM tables to text files. Additionally, files can be transferred to a remote server, one by one or as an archive.

The configuration produces a set of plan files and one workflow .ewf file executing all relevant plans and performing additional tasks like compressing and uploading files to a remote server.

Step 1 Understand export task anatomy

A text file export task has the following hierarchical structure:

Task = One workflow file.
- Export = One plan file and one exported text file.
  - Table = One Extended Reader step in the plan.

A task can include several exports, which in turn can export data from several tables at once.

Step 2 Determine your use case

The export functionality in RDM supports a few optional features, like compressing exported text files into an archive, or sending exported files to the remote server. Combinations of the available options result in the following use cases:

Export files from RDM to the local server (containing the RDM Runtime and ONE Runtime Server).
In addition to (1), compress the exported files into an archive (ZIP and GZIP formats are supported).
In addition to (1), send the exported files to a remote server.
In addition to (2), send the archive to a remote server.

Step 3 Define a synchronization task in the RDM project

Expand Synchronization > File > File (exports).
Right-click File (exports) and select New synchronization task.
Fill in Synchronization task attributes according to your use case.
Under Exports, select Add and then double-click the row number to enter export definition details.
Fill in Export definition attributes according to your use case.
Select Apply to save the current configuration and then select the left arrow in the upper-right corner.
Repeat steps 4-5 until you are done with all exports.
Select OK to close the windows.
Save changes to your project and reload metadata.

Right-click the newly created task and select Generate workflow and plan(s).

A workflow file is generated to Files > workflows and all dependent plans are generated to Files > plans > synchronization > FTs > exports.

The naming conventions are as follows:

Workflow file: [task_name]_export.ewf, for example, newtask_export.ewf.
Plans: [task_name]_[file_name]_export.plan, for example, newtask_products.txt_export.plan.

You can also generate plans separately for each exported file: right-click the node with the filename under the task and select Generate plan.

Synchronization task attributes

Field Required for use cases Description

Name

All

Task name.

Server name

3, 4

Name of a previously defined server connection.

To define a server connection, switch to the File Explorer view, right-click Servers > New Server. See one-desktop:server:connect-to-a-server.adoc.

Remote Server Target Directory

3, 4

Directory on the remote server where the file with data is uploaded to from the local server (containing the RDM Runtime and ONE Runtime Server).

Local Server Archive Directory

2, 4

Directory on the local server (containing the RDM Runtime and ONE Runtime Server), to which the archive is saved (if Compression is used).

The path is relative to your RDM project folder and should point to a descendant of the Files folder, for example, Files/data/FTs/exports.

Archive Name

2, 4

File (text or archive) to download from the remote server.

Compression

2, 4

Specifies the compression type (if any).

Exports

All

The list of subtasks that export data from one or several tables into one text file, A plan is generated for each export.

Export definition attributes

Field Required for use cases Description

File name

All

RDM exports data to a file with this name (TXT and CSV files supported).

Field separator

All

The symbol that separates columns in the text file.

Line separator

All

The symbol that marks the end of the line: \r (MacOS 9 and older) , \n (Linux and MacOs X), or \r\n (Windows).

Write header

N/A

If checked, column headers are written to the text file together with data.

Target directory

Directory on the remote server to which the file is sent.

File path

All

Directory on the local server where to save the text file.

The path is relative to your RDM project folder and should point to a descendant of the Files folder, for example, Files/data/FTs/exports.

Authentication Strategy

All

Determines the authentication configuration for the RDM Extended Reader step in the generated export plan. See App Variables for information about authentication strategies.

Generate additional steps

N/A

If set to false (default), only the RDM Extended Reader step is generated or updated according to the logical model within the generated plan (other steps stay untouched). If set to true, a set of default steps is generated or updated automatically into the plan.

Tables > Name

All

The list of tables whose data is exported within the task.

Tables > History

N/A

If checked, data from the ALL_HISTORY mode is downloaded together with the currently published. Also, a _hist suffix is added to the file name. See Data Viewing Modes for more information about the ALL_HISTORY table.

Step 4 Configure plans for exporting data

When a plan for exporting data to a text file is generated for the first time, it contains the following steps:

RDM Extended Reader times the number of exported tables - Reads data from RDM.
Text File Writer - Writes exported data to a text file.
Multiplicator - Optional step for the case of multiple RDM Extended Readers: makes input parameters available to all Extended Readers.
Sort - Optional step for sorting data.

The plan must be further configured:

Right-click a node with the filename under an export task and select Open Plan.
Add an input step (for example, a Text File Reader or JDBC Reader) containing parameters for RDM Extended Reader.

The data source should have one row of data containing the following data (the column names are up to you: they are mapped to configuration fields in the RDM Reader steps): * username (Optional: if you are passing credentials manually) - A valid RDM user with read permissions for the table. * password (Optional: if you are passing credentials manually) - User password. * timestamp - (1) The date as of which to export data; (2) In case incremental timestamp is set, the end of the period for which to export data from the ALL_HISTORY mode. * incremental timestamp (optional) - The beginning of the period for which to export data from the ALL_HISTORY mode.

+ .Example parameters file
```
username;password;timestamp;timestamp_increment
betty;5ecurePassword;;2016-01-01 00:00:00
```
+ NOTE: Leaving the timestamp attribute empty is treated as "now".

+ IMPORTANT: The date format in the example is the recommended date format.
Connect the input step with parameters to Multiplicator.
For each RDM Extended Reader step, double-click the step to configure it.

On the General tab, map the parameters from step 2 to the step configuration corresponding fields.
(Optional: if you are passing credentials manually) From the step navigation tree, select Credentials and configure the credentials:
- In the User field, enter the name of the attribute containing the username.
- In the Password field, enter the attribute containing user password.
In case of multiple RDM Extended Reader steps, join and transform their data flows as needed (for example, by using the Join steps) and connect them to the Sort step (see the example plan).
(Optional) Configure the Sort step to have the data sorted.
Double-click Text File Writer, switch to the Columns tab, and specify the columns that should be written the text file or select Write All Columns.

The plan is ready to be tested and deployed. In the following example plan, we are joining data from two RDM tables: PRODUCT and PRODUCT_GROUP, with the goal of extracting a list of all products into the name_long column and the product groups they belong to into the name_preferred column. We join the data flows by the code_prod_group column present in both tables.

Step 5 Schedule the export task

The workflow generated into Files > workflows can be scheduled for regular runs by the scheduler. See Scheduling Tasks.

Was this page useful?