If you can't find the product or version you're looking for, visit support.ataccama.com/downloads

HDFS Operate On File

Provides six potential file operations: File Exist, File Not Exist, Mkdir, Delete, Copy, and Move, which are defined in the Operation parameter.

The task evaluates both /data/tmp and /data/tmp/ paths as a tmp folder, so to operate a folder’s contents without the folder itself, we recommend using the /data/tmp/?* mask.

All operations respect permissions on the filesystem (that is, in case of insufficient permissions on the specified source or target, the task fails).

Properties

Name	Type	Description	Expression support
Operation	mandatory	Defines a file operation. Possible implementations: File Exist, File Not Exist, Mkdir, Delete, Copy, Move.	none
Hadoop Source	mandatory	Name of an existing cluster connection.	none

Name

Type

Description

Expression support

Operation

mandatory

Defines a file operation. Possible implementations: File Exist, File Not Exist, Mkdir, Delete, Copy, Move.

none

Hadoop Source

mandatory

Name of an existing cluster connection.

none

File Exist operation

Verifies the existence of a source file or folder.

Name	Type	Description	Expression support
Operation	mandatory Must be "File Exist Operation".	Operation name.	none
Target	mandatory	Path to the source file or directory. The task fails when the source does not exist.	semi-expression

Name

Type

Description

Expression support

Operation

mandatory

Must be "File Exist Operation".

Operation name.

none

Target

mandatory

Path to the source file or directory. The task fails when the source does not exist.

semi-expression

File Not Exist operation

Verifies the absence of a source file or folder.

Name	Type	Description	Expression support
Operation	mandatory Must be "File Not Exist Operation".	Operation name.	none
Target	mandatory	Path to the source file or directory. The task fails when the source exists.	semi-expression

Name

Type

Description

Expression support

Operation

mandatory

Must be "File Not Exist Operation".

Operation name.

none

Target

mandatory

Path to the source file or directory. The task fails when the source exists.

semi-expression

Mkdir operation

Creates a folder.

Name Type Description Expression support

Name	Type	Description	Expression support
Operation	mandatory Must be "Mkdir Operation".	Operation name.	none
Target	mandatory	Path to the destination folder. The task fails when the target is a multi-level directory structure from which some of the parent folders do not exist and Recursive is false.	semi-expression
Recursive	optional Default value: `false`.	Permission to make parent directories as needed.	none

Operation

mandatory

Must be "Mkdir Operation".

Operation name.

none

Target

mandatory

Path to the destination folder. The task fails when the target is a multi-level directory structure from which some of the parent folders do not exist and Recursive is false.

semi-expression

Recursive

optional

Default value: false.

Permission to make parent directories as needed.

none

Delete operation

Deletes a target file or folder. The task fails when it does not have permissions to delete Target.

Name Type Description Expression support

Name	Type	Description	Expression support
Operation	mandatory Must be "Delete Operation".	Operation name.	none
Target	mandatory	Path to the mask on the target file or folder. The task fails when no file or directory is found to mask. Supports using wildcards .	semi-expression
Recursive	optional Default value: `false`.	Permission to delete subdirectories.	none

Operation

mandatory

Must be "Delete Operation".

Operation name.

none

Target

mandatory

Path to the mask on the target file or folder. The task fails when no file or directory is found to mask. Supports using wildcards .

semi-expression

Recursive

optional

Default value: false.

Permission to delete subdirectories.

none

Copy operation

Copies the source file or folder to the destination folder. The operation automatically preserves target files timestamps.

Name Type Description Expression support

Name	Type	Description	Expression support
Operation	mandatory Must be "Copy Operation".	Operation name.	none
Source	mandatory	Path to the source mask on files or folders. The task fails when no file or directory is found to mask. Supports using wildcards.	semi-expression
Target	mandatory	Path to the destination folder or file: Folder - The file after copying observes the name; the folder must exist, otherwise the task fails. File - The parent folder of the target file must exist, otherwise the task fails.	semi-expression
Overwrite	optional Default value: `false`.	Permission to overwrite files in the destination folder. The task fails when Overwrite is `false` and some of the target files or folders already exist.	none
Recursive	optional Default value: `false`.	Permission to copy subdirectories.	none

Operation

mandatory

Must be "Copy Operation".

Operation name.

none

Source

mandatory

Path to the source mask on files or folders. The task fails when no file or directory is found to mask.

Supports using wildcards.

semi-expression

Target

mandatory

Path to the destination folder or file:

Folder - The file after copying observes the name; the folder must exist, otherwise the task fails.
File - The parent folder of the target file must exist, otherwise the task fails.

semi-expression

Overwrite

optional

Default value: false.

Permission to overwrite files in the destination folder. The task fails when Overwrite is false and some of the target files or folders already exist.

none

Recursive

optional

Default value: false.

Permission to copy subdirectories.

none

Move operation

Moves the source file or folder to the destination folder. The operation automatically preserves target files timestamps.

The task fails when it does not have permissions to delete Source.

Name Type Description Expression support

Name	Type	Description	Expression support
Operation	mandatory Must be "Move Operation".	Operation name.	none
Source	mandatory	Path to the source mask on files or folders. The task fails when no file or directory is found to mask. Supports using Wildcards in Workflow Tasks.	semi-expression
Target	mandatory	Path to the destination folder or file: Folder - After moving, the file observes the name; the folder must exist, otherwise the task fails. File - The parent folder of the target file must exist, otherwise the task fails.	semi-expression
Overwrite	optional Default value: `false`.	Permission to overwrite files in the destination folder. The task fails when Overwrite is `false` and some of the target files or folders already exist.	none
Recursive	optional Default value: `false`.	Permission to move subdirectories.	none

Operation

mandatory

Must be "Move Operation".

Operation name.

none

Source

mandatory

Path to the source mask on files or folders. The task fails when no file or directory is found to mask.

Supports using Wildcards in Workflow Tasks.

semi-expression

Target

mandatory

Path to the destination folder or file:

Folder - After moving, the file observes the name; the folder must exist, otherwise the task fails.
File - The parent folder of the target file must exist, otherwise the task fails.

semi-expression

Overwrite

optional

Default value: false.

Permission to overwrite files in the destination folder. The task fails when Overwrite is false and some of the target files or folders already exist.

none

Recursive

optional

Default value: false.

Permission to move subdirectories.

none

Was this page useful?