Data Quality Workbench User's Guide > Data Quality Workbench Components Configuration Functionality Overview > Filtering: General Approach and Global Filters

Filtering: General Approach and Global Filters

Topics:

You can add a global filter by clicking Add Filter on the upper pane of the screen and using the context menus.

A filter can be added from the Profile and Data Table pages (from the table value and table header). If you added a filter from the table header, then the focus will be on the filter value, where you can enter the data immediately.

Clicking the Add Filter button on the screen creates a filter form where you can populate the Property and Value. User can also change the Operator, which is pre-populated with the first value in the drop-down list.

By default, the filter consists of the following fields:

In the Properties drop-down list, the whole list of Model subjects and their attributes are available.

The first attribute and operator are set as default. For some properties, the filter will consist of two fields: Is empty, and Is not empty. However, some values will be prepopulated (for example, for the Boolean attribute type, the value will be set to True). Attributes and subjects are displayed in the drop-down list according to their location, but currently, only attributes can be selected.

Operators are different depending on the type of attribute selected:

Once you have added a filter with a criteria, the Profile page, data table, and onscreen filter will be recalculated and data records will be displayed according to the added filter.

If you click on the X icon on the right side of the Filter pane, added filter is removed.

You can add multiple filter tags by combining them with the AND/OR operands. The first two added filter tags can use the AND operand. Filters separated with the OR operand are combined on the user interface and can not add new filters between them. Multiple filters can be added from the table or by the add (+) button in the global filter area. New filters can be inserted after each filter tag:

Values will be filtered according to the added filters and operands. If you combine filters for different attributes with the AND operand, then records that match all filter criteria will be displayed.

Filters separated with the OR operand are combined on the user interface where you can not add new filters between them. If you combine filters with the OR operand, then records that match at least one of the filtered criteria will be displayed.

Filters are saved only per one browser session, until you refresh the page or log out.

Filtering by Patterns

The Pattern (a regular expression – RegEx) is a special text string for describing a filter pattern. You can think of regular expressions as wildcards. Patterns are used when you need to filter by some general criteria: letters and digits, four letters and two digits, word and number. The general idea is to simplify filtering.

  • If you are filtering data using the On-Screen filter/Patterns tab, you should select the bar chart that corresponds to what you need.

    For example, if you want to filter all names consisting of five letters like "Megan", you should select LLLLL as the filter criteria, and for filtering SSN "457-55-5462", you should select DDD-DD-DDDD.

  • If you want to filter data using a global filter, you need to transform the filter criteria to the RegEx format since Filtering by patterns works only when the RegEx format is entered:

The Like/Not like operators are used for filtering by patterns, which are available only for string values.

Patterns will be transformed to the RegEx format in the following ways:

There are several types of data patterns: L/D, L1/D1, and W/N patterns.

L/D Patterns (Letters/Digits):

  1. Change all L instances to [a-z]
  2. Change all D instances to [0-9]

For example, if you want to filter some specific value Iris45 you should enter [a-zA-Z][a-z][a-z][a-z][0-9][0-9] as filter criteria.

L[1]/D[1] Patterns

  1. Change all L[x] to [a-z]{x}
  2. Change all D[x] to [0-9]{x}

{x} means the quantity of Letters or Digits. {x} should not be written if there is a single digit or letter.

For example, to filter the same value Iris45, you can also transform [a-z]{4}[0-9]{2} as filter criteria.

For example, you can transform Iris4 as [a-zA-Z]{4}[0-9].

W/N Patterns

  1. Change all W to [a-z]+
  2. Change all N to [0-9]+ Patterns
  3. Change all L to [a-z]
  4. Change all D to [0-9]

For example, to transform Iris45, you should enter [a-zA-Z]+[0-9]+.

Transforming to RegEx should be used according to the case level (upper/lower).

If you want to filter records using wildcards or by using special characters (. ? + * | { } [ ] ( ) " \) – put the backslash (\) before the required special character.

For example, to filter patterns such as ID123*a?, you should enter [A-Z]{2}[0-9]{3}\*[a-z]\?.

Quick Filter

You can add any item from the data table to the global filter by right-clicking and selecting Add to filter. The global filter is enhanced with a new Filter tag where the property = <column in which the cell user clicked on lies>, operand = "Equals", and value = <actual value of the cell user clicked on>.

On-Screen Filter

The On-Screen filters panel is a panel with filters combined in groups. The On-Screen filters panel is located on Data Tables page and is displayed only for the selected table column.