Configure the Profiling Step

The Basic tab contains fields for specifying the step name, output file name and location, and default locale for the generated files.

The Masks tab contains the masks that have been defined. This is also where you can create and edit masks.

A mask is a way of showing the structure of the data rather than the content. To describe patterns in the data, user-defined codes are applied instead of the actual characters.

For example, the mask "D" could be used to represent a digit, while "L" could be used to represent a letter. The codes and rules for the masks can be defined here.

You can assign any of the created masks to the columns in your data on the Data tab for each of your inputs. See Data.

Each mask configuration contains the following settings:

To mask all characters individually regardless of the sequence left, leave the fields Repeated Symbol and Repeated Threshold blank.

To create a new mask, select the plus sign (+) at the bottom of the list of masks. To delete a mask, use the minus sign (-).

A mask can contain multiple masking rules (called character groups), as shown previously. To add a new character group, select Add.

The order of character groups might be important when creating a mask with several character groups, one of which is a subset of another, especially when the [:all:] character type is used in conjunction with other types. For example, if you want to mark all letters with "L", all digits with "D", and any other symbol with a dollar sign ("$"), the rule containing [:all:] should be placed last in the list. Use To Top, Up, Down, and To Bottom options to change the placing of rules relative to each other.

The Drill-through tab specifies whether the drill-through functionality is used or not. Enabling drill-through allows to inspect the individual records that comprise the generated statistics and other measures shown in the Profile viewer.

Because it stores additional fields, it requires a database connection for use.

To allow drill-through in the generated profile, select Enable Drill-through.

The following parameters are available:

When drill-through is enabled, you can use it by right-clicking many of the statistics shown in the Profile Viewer (for example, median value) and selecting Drill-through.

Foreign key analysis can be performed when there are two or more different inputs connected to the same Profiling step. To learn how to add several inputs to the Profiling step, see Input category.

To create a new foreign ley analysis:

To learn more about how to read foreign key analysis results, see Advanced Analyses, section Foreign keys.

The Data tab shows all the data that will be profiled. It also contains the individual settings for each column that should be profiled.

By default, the settings that are defined in the Create Profile wizard apply to all columns, but this tab allows configuring each column separately.

The Data tab has the following configuration possibilities:

When connected to an incoming step (such as Text File Reader), you can use the Fill Columns option to automatically add columns that are connected to the input rather than manually type the incoming column names. The Type column uses the types defined in the originating input step (such as Text File Reader) and is for reference only.

To change the data type of a column, use the Metadata Editor or input step.

The Dependencies tab can be used to define an analysis to test the dependency of the fields in one column on the fields in other columns, such as whether the birth number is related in some way to the date of birth.

To add a new dependency, select the plus sign (+). To delete a dependency, use the minus sign (-).

A dependency configuration consists of three parts:

The following table demonstrates several dependencies scenarios.

For more details about the dependency analysis, see the Profiling step documentation in the product Help (Help > Help Contents).

To learn more about how to read dependency analysis results, see Advanced Analyses, section Dependency Analysis.

A roll up is a way of looking at a specific subset of the profiled data. Entering a column name in the Roll Ups Expression field creates a list of separate profile analyses for each value of that column.

For example, if the data contains a column of genders, creating a roll up of this column allows you to view profile results for each gender value in that column, including empty rows, which might reveal new patterns in the data. When a roll up is defined, the Inputs and Roll Ups panel is shown in the Profile Viewer.

To learn more about how to read results per roll up, see Read a Data Quality Profile, section Inputs and Roll Ups.

A business rule is a Boolean expression that is evaluated and its results are presented in the Profile Viewer. Some examples are shown in this section.

For a description of available expressions and their usage, see ONE Expressions.

To learn more about how to read business rule results, see Advanced Analyses, section Business Rules.

To analyze the uniqueness of a particular column and determine whether it is a primary key, add a new primary key analysis and enter the name of the column to analyze in the Expression field of the Components section. Several columns can be analyzed in conjunction as seen in the following example:

To learn more about how to read primary key analysis results, see Advanced Analyses, section Primary keys.