Lead your team forward
OCT 24 / 9AM ET Register nowLookup Items
Lookup Files allow you to use Reference Data in ONE – they provide a list of predefined values for an attribute, and can be used both in DQ Evaluation rules and in Detection rules.
Lookup files can be found in Data Quality > Lookup Items.
Create lookup item
-
In Data Quality > Lookup Items select Create.
-
Provide general information about the lookup item, such as name and description (optional).
Provide descriptive names, or descriptions, to make it easier to locate the lookup in MinIO. -
In Source item select the catalog item containing the attribute from which you would like to build the lookup values. After selecting the source catalog item, you see a data preview of the first 50 lines of the selected catalog item.
-
In Attribute select the attribute from which you would like to build the lookup values
You can create lookups directly from catalog items and attributes, in which case this information is pre-filled for you.
You can do this from the catalog item view from multiple tabs, including Overview, Data, and Profile & DQ insights, or from the attribute view. Use the three dots icon and select Create new lookup.
-
In Setup, define the following:
-
Duplicities handling:
-
First: One duplicated value is permitted for each value in the lookup item. The one allowed duplicate is the first occurrence of the duplicated value.
-
Omit: No duplicated values are accepted in lookup items.
-
Accept: All duplicates are accepted and there is no limit to the number of repeated values in a lookup item.
-
-
Matching values:
-
Remove accents: If selected, removes all diacritic marks (accents).
-
Remove repeated characters: If selected, removes repeated characters.
-
Squeeze whitespaces: If selected, replaces repeated white space characters with a single space character.
-
Upper case: If selected, converts all words to upper case.
-
-
Supported characters: Define the supported characters using the format
[:letter:] [:number:]
.If defined, characters not contained in this set are removed or substituted. By default, all characters are supported (
[:all:]
). -
Substitute with: Define a string to substitute any unsupported characters, if you have defined specific supported characters in Supported Characters.
-
-
Select Save.
-
Select Build Lookup.
-
The lookup is built. You can now edit lookup metadata, manage access, view historical versions, and edit, delete, or update the lookup.
Edit
In ONE, navigate to Knowledge Catalog > Data Quality > Lookup Items. In the three dots menu of the lookup item you want to modify, select Edit or Delete accordingly.
The editing options available depend on the lookup item type. Lookup items that have been uploaded can not be updated within ONE Web Application, only reuploaded. See Upload lookups.
Lookups can also be built as a result of rule configuration.
This happens when the user selects the System-created lookups, that is, those that were created automatically during rule implementation, are also listed on the lookups screen. It is possible to edit them but changes made here can break the rule. |
Upload lookups
To upload an updated version of a lookup item:
-
Navigate to Data Quality > Lookup Items.
-
Select Upload.
-
Use the file explorer to search for lookup files (
.lkp
). -
Select and Open. Once the lookup has been uploaded, it is in draft state. Publish the page to complete the update.
Lookup items that have been uploaded can not be updated within ONE Web Application. When a lookup item is reuploaded, a second lookup is created. It is necessary to delete the original lookup if required. |
Storage
Any lookups in ONE are part of OneFileSystem
in MinIO.
Specifically, they are stored in the bucket LOOKUPS
.
Lookup files in ONE Desktop
Lookup files contain reference data that can help with the cleansing, verification, matching and enrichment of records by comparing raw data records to formatted and cleansed records.
Lookup files are an integral part of data quality and can be created in ONE Desktop using the Lookup Builder step, and incorporated into plans using the Lookup Reader step.
Lookup files built using the Lookup Builder step in ONE Desktop are available in Files > data > out > [name].lkp
, and can then be uploaded to ONE.
Use lookups in rules
While lookups can be built as a result of rule creation (when using the is from catalog item and is not from catalog item options), they can also be used in rule implementation.
To utilize lookups in rule conditions, use the options is from reference data and is not from reference data.
If you are using Advanced Expression and not Condition Builder, use the syntax Alternatively, use the |
Was this page useful?