ONE Desktop ONE DQG ONE MDM ONE RDM ONE Runtime Server
latest (15.1.0) 14.5.x 13.9.x 13.8.x 13.7.x 13.6.x 13.5.0 13.4.x 13.3.x 13.2.x
User Community Service Desk Downloads
If you can't find the product or version you're looking for, visit support.ataccama.com/downloads

Iterate

Iterate icon

This task is used to implement a for-each cycle. It sequentially runs child workflows for each input value (for example, a file or database row).

The task writes the set of retrieved values (files, database rows) into the memory immediately after being launched and runs child workflows based on that set, ignoring any filesystem or database changes that might affect the original set of values.

Properties

Name Type Description Expression support

Iterable

mandatory

Implementation and configuration of additional iterators. Possible implementations: File Iterator, Set Iterator, SQL Row Iterator.

none

Workflow Id

mandatory

ID of a workflow to run. Workflow ID specification depends on the workflow execution context:

  • In the server context, use the <sourceId>:<workflowName>.ewf notation, for example, WF02:02_03b_Executor.ewf. The sourceId prefix is defined in Workflow Component.

  • Outside the server (from IDE or via runewf.[bat|sh]), you can use two mutually exclusive notations:

    • <sourceId>:<workflowName>.ewf: In this case, the <sourceId>: prefix is ignored and the specified workflow from the current folder is run. This is useful when you want to develop and test workflows locally in ONE Desktop and deploy them to the server later on.

    • <path>/<workflowName>.ewf: For example, ../02_Run_Plans/02_03b_Executor.ewf. The path is optional and relative to the current workflow.

semi-expression

Iteration Type

mandatory

SERIAL or PARALLEL run.

none

Break On Error

optional; Default value: false.

If an error occurs, the iteration is either terminated immediately (true) or continues (false).

none

The task finishes with the result OK, unless any of the child workflows does not start correctly or finishes with a wrong result. The Break On Error parameter has no impact on the task result. When the iterator has no values, the task finishes with the result OK.

File Iterator

The task runs child workflows for all files in the Directory.

When Iteration Type is set to SERIAL, the task iterates in the order corresponding to the alphabetical order of files specified in Directory and Mask.

Properties

Name Type Description Expression support

Directory

mandatory

Location of files. The directory can be local, on Amazon S3 or Azure Data Lake Storage Gen1 server, and on HDFS (if your product contains ONE Spark DPE). The remote resources are accessible with the resource://<resourceName>/<path>/<inputFile> syntax.

semi-expression

Mask

mandatory

File name mask used to select files to iterate over.

Supports the following wildcards:

  • ?: One character, except file separators forward slash (/) and backslash (\).

  • *: Zero to n characters, except file separators forward slash (/) and backslash (\).

semi-expression

Iterable Item

mandatory

Name of the variable that is passed to the child workflow (that is, a variable with the same name should be defined in the child workflow as an input variable). The variable value would be the processed file name (including full path) in each iteration.

none

Continue on error Flag

optional

Default value: false.

When file or folder information cannot be used, the task reports to log and continues, it does not fail.

none

Recursive

optional

Default value: false.

Files in all subdirectories of the directory are either used for iteration too (true) or not (false).

none

Condition

optional

Expression evaluated for each candidate entry to check if it should be included in the output or not.

Supports the following variables:

Name Expression type Description

name

string

Name of the file or folder.

fullName

string

Complete path of the file or folder.

isDirectory

boolean

Set to true if item is a directory, false if it is file.

size

long

Size of the file or folder in bytes.

timestamp

date/time

Last modification date and time.
Not all file systems return all the properties so some of them might be empty.

expression

Parameter Mapping

optional

Set of user defined parameters to pass to the workflow as global variable values.

Parameter mapping properties

Name Type Description Expression support

Parameters

optional

Set of user defined parameters to pass to the workflow as global variable values.

none
Parameters properties
Name Type Description Expression support

Name

mandatory

Name of the parameter.

semi-expression

Expression

mandatory

Expression evaluating the value of the parameter.

expression

Example

Sample file iterator
<!-- File iterator with the mask property defined using a constant string -->
<iterable>
  <directory>./inputs</directory>
  <iterableItem>IterationValue</iterableItem>
  <mask>*.csv</mask>
  <parameterMapping>
    <parameters>
      <parameter expression="'some_text'" name="childrenVar01"/>
      <parameter expression="var01" name="childrenVar02"/>
    </parameters>
  </parameterMapping>
  <recursive>true</recursive>
</iterable>

<!-- File iterator with the mask property defined using a variable -->
<iterable>
  <directory>./inputs</directory>
  <iterableItem>IterationValue</iterableItem>
  <mask>${FILE_MASK}</mask>  <!-- if the FILE_MASK variable contains for example a value "*.txt", the task will iterate over all .txt files found in the defined directory -->
  <parameterMapping>
    <parameters>
      <parameter expression="'some_text'" name="childrenVar01"/>
      <parameter expression="var01" name="childrenVar02"/>
    </parameters>
  </parameterMapping>
  <recursive>true</recursive>
</iterable>

Set iterator

The task runs child workflows for all items defined in the Set.

When Iteration Type is set to SERIAL, the task iterates in the order corresponding to the specified order of items defined in Set.

Properties

Name Type Description Expression support

Set

mandatory

Set to be used for iteration.

semi-expression

Separator

mandatory

Value of separator that is used.

none

Iterable Item

mandatory

Name of the variable that is passed to the child workflow (that is, a variable with the same name should be defined in the child workflow as an input variable). The value of the variable is taken from the Set in each iteration.

For example, if you define Set as a 1,b 2 and the Separator as a comma (,) the task iterates two times and passes the following values to the child workflow: a 1 and b 2.

none

Parameter Mapping

mandatory

Set of user defined parameters to pass to the workflow as global variable values.

Parameter mapping properties

Name Type Description Expression support

Parameters

optional

Set of user defined parameters to pass to the workflow as global variable values.

none
Parameters properties
Name Type Description Expression support

Name

mandatory

Name of the parameter.

semi-expression

Expression

mandatory

Expression evaluating the value of the parameter.

expression

Example

Sample set iterator
<iterable>
  <iterableItem>IterationValueName</iterableItem>
  <parameterMapping>
    <parameters>
      <parameter expression="'some_text'" name="childrenVar01"/>
      <parameter expression="var01" name="childrenVar02"/>
    </parameters>
  </parameterMapping>
  <separator>,</separator>
  <set>1,2,5</set>
</iterable>

SQL row iterator

The task runs child workflows for each row returned by an SQL query.

When Iteration Type is set to SERIAL, the iteration order corresponds to the order of results returned by Query (so ORDER BY clause can be used to affect the iteration order).

Properties

Name Type Description Expression support

Connection Name

mandatory

Connection name to use for query invocation.

semi-expression

Query

mandatory

Query to execute.

semi-expression

Mapping

optional

Set of parameters derived from the result set to pass to the workflow as global variable values.

none

Parameter Mapping

optional

Set of user defined parameters to pass to the workflow as global variable values.

Mapping properties

Name Type Description Expression support

Result Set

optional

Defines a set of output parameters to be read from the query’s result set. Applicable when running SQL queries.

none
Result set properties
The value is read from the first row of the first available result set.
Name Type Description Expression support

Name

mandatory

Defines the name of the mapping. The value read from the mapping is available under this name in the condition and saved in the task variables.

none

Index

mandatory

Defines the index of the value in the result set. Indexed from 1 (that is, index of the first available column is 1).

none

Parameter mapping properties

Name Type Description Expression support

Parameters

optional

Set of user defined parameters to pass to the workflow as global variable values.

none
Parameters properties
Name Type Description Expression support

Name

mandatory

Name of the parameter.

semi-expression

Expression

mandatory

Expression evaluating the value of the parameter.

expression

Example

Sample SQL row iterator
<iterable>
  <connectionName>derby</connectionName>
  <mapping>
    <resultSet>
      <resultSetMapping name="IterationValue" index="2"/>
    </resultSet>
  </mapping>
  <parameterMapping>
    <parameters>
      <parameter expression="'some_text'" name="childrenVar01"/>
      <parameter expression="var01" name="childrenVar02"/>
    </parameters>
  </parameterMapping>
  <query>select * from Table</query>
</iterable>

Was this page useful?