Lead your team forward
OCT 24 / 9AM ET Register nowConfiguring Tasks for Exporting Data to Text Files
The following steps explain how to configure a task for exporting data from RDM tables to text files. Additionally, files can be transferred to a remote server, one by one or as an archive.
The configuration produces a set of plan files and one workflow .ewf
file executing all relevant plans and performing additional tasks like compressing and uploading files to a remote server.
Step 1 Understand export task anatomy
A text file export task has the following hierarchical structure:
-
Task = One workflow file.
-
Export = One plan file and one exported text file.
-
Table = One Extended Reader step in the plan.
-
-
A task can include several exports, which in turn can export data from several tables at once.
Step 2 Determine your use case
The export functionality in RDM supports a few optional features, like compressing exported text files into an archive, or sending exported files to the remote server. Combinations of the available options result in the following use cases:
-
Export files from RDM to the local server (containing the RDM Runtime and ONE Runtime Server).
-
In addition to (1), compress the exported files into an archive (ZIP and GZIP formats are supported).
-
In addition to (1), send the exported files to a remote server.
-
In addition to (2), send the archive to a remote server.
Step 3 Define a synchronization task in the RDM project
-
Expand Synchronization > File > File (exports).
-
Right-click File (exports) and select New synchronization task.
-
Fill in Synchronization task attributes according to your use case.
-
Under Exports, select Add and then double-click the row number to enter export definition details.
-
Fill in Export definition attributes according to your use case.
-
Select Apply to save the current configuration and then select the left arrow in the upper-right corner.
-
Repeat steps 4-5 until you are done with all exports.
-
Select OK to close the windows.
-
Save changes to your project and reload metadata.
-
Right-click the newly created task and select Generate workflow and plan(s).
A workflow file is generated to Files > workflows and all dependent plans are generated to Files > plans > synchronization > FTs > exports.
The naming conventions are as follows:
-
Workflow file:
[task_name]_export.ewf
, for example,newtask_export.ewf
. -
Plans:
[task_name]_[file_name]_export.plan
, for example,newtask_products.txt_export.plan
.
You can also generate plans separately for each exported file: right-click the node with the filename under the task and select Generate plan. -
Synchronization task attributes
Field | Required for use cases | Description | ||
---|---|---|---|---|
Name |
All |
Task name. |
||
Server name |
3, 4 |
Name of a previously defined server connection.
|
||
Remote Server Target Directory |
3, 4 |
Directory on the remote server where the file with data is uploaded to from the local server (containing the RDM Runtime and ONE Runtime Server). |
||
Local Server Archive Directory |
2, 4 |
Directory on the local server (containing the RDM Runtime and ONE Runtime Server), to which the archive is saved (if Compression is used).
|
||
Archive Name |
2, 4 |
File (text or archive) to download from the remote server. |
||
Compression |
2, 4 |
Specifies the compression type (if any). |
||
Exports |
All |
The list of subtasks that export data from one or several tables into one text file, A plan is generated for each export. |
Export definition attributes
Field | Required for use cases | Description | ||
---|---|---|---|---|
File name |
All |
RDM exports data to a file with this name (TXT and CSV files supported). |
||
Field separator |
All |
The symbol that separates columns in the text file. |
||
Line separator |
All |
The symbol that marks the end of the line: |
||
Write header |
N/A |
If checked, column headers are written to the text file together with data. |
||
Target directory |
3 |
Directory on the remote server to which the file is sent. |
||
File path |
All |
Directory on the local server where to save the text file.
|
||
Authentication Strategy |
All |
Determines the authentication configuration for the RDM Extended Reader step in the generated export plan. See App Variables for information about authentication strategies. |
||
Generate additional steps |
N/A |
If set to |
||
Tables > Name |
All |
The list of tables whose data is exported within the task. |
||
Tables > History |
N/A |
If checked, data from the ALL_HISTORY mode is downloaded together with the currently published.
Also, a |
Step 4 Configure plans for exporting data
When a plan for exporting data to a text file is generated for the first time, it contains the following steps:
-
RDM Extended Reader times the number of exported tables - Reads data from RDM.
-
Text File Writer - Writes exported data to a text file.
-
Multiplicator - Optional step for the case of multiple RDM Extended Readers: makes input parameters available to all Extended Readers.
-
Sort - Optional step for sorting data.
The plan must be further configured:
-
Right-click a node with the filename under an export task and select Open Plan.
-
Add an input step (for example, a Text File Reader or JDBC Reader) containing parameters for RDM Extended Reader.
The data source should have one row of data containing the following data (the column names are up to you: they are mapped to configuration fields in the RDM Reader steps): * username (Optional: if you are passing credentials manually) - A valid RDM user with read permissions for the table. * password (Optional: if you are passing credentials manually) - User password. * timestamp - (1) The date as of which to export data; (2) In case incremental timestamp is set, the end of the period for which to export data from the ALL_HISTORY mode. * incremental timestamp (optional) - The beginning of the period for which to export data from the ALL_HISTORY mode.
+ .Example parameters file
username;password;timestamp;timestamp_increment betty;5ecurePassword;;2016-01-01 00:00:00
+ NOTE: Leaving the
timestamp
attribute empty is treated as "now".+ IMPORTANT: The date format in the example is the recommended date format.
-
Connect the input step with parameters to Multiplicator.
-
For each RDM Extended Reader step, double-click the step to configure it.
On the General tab, map the parameters from step 2 to the step configuration corresponding fields.
-
(Optional: if you are passing credentials manually) From the step navigation tree, select Credentials and configure the credentials:
-
In the User field, enter the name of the attribute containing the username.
-
In the Password field, enter the attribute containing user password.
-
-
In case of multiple RDM Extended Reader steps, join and transform their data flows as needed (for example, by using the Join steps) and connect them to the Sort step (see the example plan).
-
(Optional) Configure the Sort step to have the data sorted.
-
Double-click Text File Writer, switch to the Columns tab, and specify the columns that should be written the text file or select Write All Columns.
The plan is ready to be tested and deployed.
In the following example plan, we are joining data from two RDM tables: PRODUCT and PRODUCT_GROUP, with the goal of extracting a list of all products into the name_long
column and the product groups they belong to into the name_preferred
column.
We join the data flows by the code_prod_group
column present in both tables.
Step 5 Schedule the export task
The workflow generated into Files > workflows can be scheduled for regular runs by the scheduler. See Scheduling Tasks.
Was this page useful?