Filters
ONE Data lets you filter attributes based on record value or data quality. To access filtering options, open your ONE Data table and select Filter.
Alternatively, if you already know which attribute you want to filter, right-click the attribute header and select Filter. This preselects the attribute as the first filtering condition.
|
Move the filtering menu around to better fit your screen. You can also close it as long as you stay on the same screen: this won’t remove the filters, only hide the filtering menu. While filtering is applied, you can consult the whole data set at any point. Use the pause icon to temporarily remove filtering, then jump back in where you left off by turning it back on (the play icon). 
To remove the filters entirely, open the Filter menu and select Clear all. |
How to apply filters
If you want to filter based on the record value, you need to define the following for each filtering condition:
-
Attribute: Select the attribute name.
-
Condition: Select the condition. The options available depend on the data type of the input attribute. Here are some of the most common ones:
-
String:
contains,begins with,ends with,is empty, `is not empty,equal to,not equal to. -
Integer:
=,>,<,>=,⇐,!=,is empty,is not empty. -
Date:
equal to,not equal to,after,before,after or same as,before or same as,is empty,is not empty. -
Boolean:
is,is empty,is not empty. -
Expression: Enter the value against which the condition is checked.
-
| If you’re filtering by DQ results, a different set of options is available. To learn more, see How to filter by data quality. |
Filters are applied as soon as you define the first condition. You can keep adding conditions (*Add condition* or the plus sign (``) in the condition row) until you get the records you’re looking for.
Make sure you select the correct operator between conditions: AND or OR.
For more complex queries, you can aggregate several conditions into one to further refine the search results (+Add group).
While filters are applied, you can quickly modify the filtered records. Right-click the attribute header to copy (Duplicate), remove (Delete), or change all filtered records in the selected column (Bulk edit).
To remove the whole set of filtered records, select the checkbox on the attribute header, then click Delete in the ribbon that appears.
| If you’re using filters to find records that you want to edit or delete in bulk, keep in mind that any previously selected records will not be modified. |
How to filter by data quality
In addition to filtering based on actual record values, you can also filter records by data quality. Data quality-based filters behave similarly to other filtering options and can be combined with them as needed. You can also configure multiple conditions that check the record data quality. For more details, see the previous section How to apply filters.
When you update filtered records to fix the detected DQ issues, records are validated on the fly and removed from your filtering results if they no longer fit the criteria. Additionally, a warning is displayed, informing you that the overall quality of your table might be outdated.
| Currently, this doesn’t happen if records are validated using multi-input rules. In that case, you need to rerun DQ evaluation to view the correct DQ results. |
When filtering by data quality, filtering conditions are defined as follows:
-
Attribute: Select the attribute name, Any attributes, or All attributes.
-
Any attributes: Filters records where at least one attribute matches the filtering conditions.
-
All attributes: Filters records where all attributes match the filtering conditions.
-
-
Condition: Choose between
failedorpassed. If you opted to filter by a specific attribute, these options are shown along with other available conditions. -
DQ rules: Here you need to select what rules a record must pass or fail to appear in filtering results.
To quickly find the DQ rules you’re looking for, you can use the following:
-
Filter DQ rules: Search by rule name, then select the rules you want to use.
-
Filter by DQ score: Select this to filter rules and conditions based on the DQ score, if configured. This doesn’t filter the data itself, only rules and conditions.
If enabled, in Conditions with score, choose the operator from the list and enter the DQ score. Select the rules you want to use from the results returned.
-
DQ dimensions: Select one, several, or all dimensions to find the rules with the matching dimensions. In this case, once you choose the dimension, all filtered rules are automatically selected for you. Add other conditions or manually clear the rules you don’t want to use in filtering.
If a rule has more than one condition configured, you can choose to filter records based only on some of them.
To do this, expand the rule and select all rule conditions that apply. Hover over the condition to see the DQ score of the condition.
-
Filter by multiple conditions
If you are applying multiple conditions, note that some records might be unexpectedly excluded from your results. This can happen due to how DQ evaluation currently works.
When a rule is applied to a record, the record is evaluated against the rule conditions starting from the first condition defined in the rule until it either passes all conditions or fails one of them. As a record is labeled as invalid if only one condition fails, once this happens, any remaining conditions are skipped.
DQ filters retain only the first condition that a record failed while skipped conditions are treated as passed, which could lead to incorrect filtering results.
Let’s take a look at the following example.
There are three conditions verifying the validity of emails: NOREPLY_EMAIL, EMPTY_EMAIL, INVALID_FORMAT.
If a record has both incorrect formatting and contains a NOREPLY string, it will not be shown if you filter all records with incorrect formatting.
The record will be labeled as failed immediately after applying the NOREPLY_EMAIL condition and the INVALID_FORMAT condition will not be verified.
Was this page useful?