If you can't find the product or version you're looking for, visit support.ataccama.com/downloads

Lookup Items

Lookup Files allow you to use Reference Data in ONE – they provide a list of predefined values for an attribute, and can be used both in DQ Evaluation rules and in Detection rules.

Lookup files can be found in Data Quality > Lookup Items.

Create lookup item

In Data Quality > Lookup Items select Create.
Provide general information about the lookup item, such as name and description (optional).

Provide descriptive names, or descriptions, to make it easier to locate the lookup in MinIO.
In Source item select the catalog item containing the attribute from which you would like to build the lookup values. After selecting the source catalog item, you see a data preview of the first 50 lines of the selected catalog item.

In Attribute select the attribute from which you would like to build the lookup values

You can create lookups directly from catalog items and attributes, in which case this information is pre-filled for you.

You can do this from the catalog item view from multiple tabs, including Overview, Data, and Profile & DQ insights, or from the attribute view. Use the three dots icon and select Create new lookup.

In Setup, define the following:
1. Duplicities handling:
  - First: One duplicated value is permitted for each value in the lookup item. The one allowed duplicate is the first occurrence of the duplicated value.
  - Omit: No duplicated values are accepted in lookup items.
  - Accept: All duplicates are accepted and there is no limit to the number of repeated values in a lookup item.
2. Matching values:
  - Remove accents: If selected, removes all diacritic marks (accents).
  - Remove repeated characters: If selected, removes repeated characters.
  - Squeeze whitespaces: If selected, replaces repeated white space characters with a single space character.
  - Upper case: If selected, converts all words to upper case.
3. Supported characters: Define the supported characters using the format [:letter:] [:number:].
  
  If defined, characters not contained in this set are removed or substituted. By default, all characters are supported ([:all:]).
4. Substitute with: Define a string to substitute any unsupported characters, if you have defined specific supported characters in Supported Characters.
Select Save.
Publish the draft.
Select Build Lookup.
The lookup is built. You can now edit lookup metadata, manage access, view historical versions, and edit, delete, or update the lookup.

Edit

In ONE, navigate to Knowledge Catalog > Data Quality > Lookup Items. In the three dots menu of the lookup item you want to modify, select Edit or Delete accordingly.

The editing options available depend on the lookup item type. Lookup items that have been uploaded can not be updated within ONE Web Application, only reuploaded. See Upload lookups.

Lookups can also be built as a result of rule configuration. This happens when the user selects the is (not) from catalog item option when defining rule logic.

System-created lookups, that is, those that were created automatically during rule implementation, are also listed on the lookups screen. It is possible to edit them but changes made here can break the rule.

Upload lookups

To upload an updated version of a lookup item:

Navigate to Data Quality > Lookup Items.
Select Upload.
Use the file explorer to search for lookup files (.lkp).
Select and Open. Once the lookup has been uploaded, it is in draft state. Publish the page to complete the update.

Lookup items that have been uploaded can not be updated within ONE Web Application. When a lookup item is reuploaded, a second lookup is created. It is necessary to delete the original lookup if required.

Storage

Any lookups in ONE are part of OneFileSystem in MinIO. Specifically, they are stored in the bucket LOOKUPS.

Lookup files in ONE Desktop

Lookup files contain reference data that can help with the cleansing, verification, matching and enrichment of records by comparing raw data records to formatted and cleansed records.

Lookup files are an integral part of data quality and can be created in ONE Desktop using the Lookup Builder step, and incorporated into plans using the Lookup Reader step.

Lookup files built using the Lookup Builder step in ONE Desktop are available in Files > data > out > [name].lkp , and can then be uploaded to ONE.

Use lookups in rules

While lookups can be built as a result of rule creation (when using the is from catalog item and is not from catalog item options), they can also be used in rule implementation.

To utilize lookups in rule conditions, use the options is from reference data and is not from reference data.

If you are using Advanced Expression and not Condition Builder, use the syntax isInFile to reference lookup files. Use isInFile and the name of the lookup in the format lookup:<name>, for example: (isInFile(ATTRIBUTE_1, "lookup:city")).

Alternatively, use the <lookupItemId>, where <lookupItemId> is the ID visible in the browser address. For example: (isInFile(ATTRIBUTE_1, "lookup:e1326a77-ee1b-4d57-acf5-55fe990d731e")).

Was this page useful?