Manual Cleansing Page

Topics:

Manual Cleansing issues involve the resolution of problems with missing or invalid data and the review of changes suggested by an automated data quality process (like incorrect, ambiguous, missing data etc.).

A cleansing case consists of a record which attributes require manual editing. Each edit may be sent through workflow separately: different attributes of the record may be in different states.

Moreover, the User may leave a comment on particular attribute or view and resolve issues / errors that arrive from automated cleansing process.

Navigating the Manual Cleansing Page

There are three ways to navigate to the Manual Cleansing page:

  1. From the Master Comparison page, to create new cleansing case via 'Edit' menu item.
  2. Through the link from Issues pop up, to view the Manual Cleansing case, which has already been created (appropriate icons are displayed on Details 360, Master Comparison, and Home pages).
  3. From the Issues page, to view the Manual Cleansing case, which has already been created.

Note: Manual Cleansing case can be opened only for Instance records. Any record currently under Manual Cleansing remediation may not be remediated under any other Manual Cleansing case.

If a user wants to open cleansing case, the user should navigate to the Master Comparison page of that record, expand Instance records section and select "Edit" menu item for the appropriate Instance.

Another way to navigate to Manual Cleansing page is via links in Issue pop up (if cleansing case already exists in the system).

In order to see the pop up user should mouse roll over any cleansing remediation icons that are displayed on Details 360, Master Comparison, and Home pages and then click on appropriate link.

Note: If a user has no permissions to view or work with cleansing cases, then the cleansing link is disabled.

If a case is assigned to the current user, the Manual Cleansing page will be displayed in Update mode, so the user can continue working with it. If not, Display mode, where the user can make no changes.

And the last way to navigate to already created cleansing case is to select appropriate link on Issues page.

The same logic applies here as with links in pop ups. If a case is assigned to the current user, then the Manual Cleansing page will be displayed in Update mode, so that the user can continue working with it. If not, then Display mode, where the user can make no changes.

Note: closed Cleansing cases are displayed on Issues page and in Remediation pop ups. Those cases can be opened in Display mode only.

Page Structure

The Manual Cleansing Case page is composed of the following main blocks:

  1. Breadcrumbs
  2. Workspace
  3. Expand / Collapse Workspace control
  4. Activity Stream
  5. Case Properties

Using breadcrumbs (1) user is able to navigate to previously visited pages.

In the Workspace (2) a User can edit the values of attributes, add comments to attributes and send the changes through workflow.

For manual cleansing case in the Workspace the Instance record attributes are displayed as well as Edited values column and workflow actions. Workspace panel's fields can be configured via Administration module.

By default workspace is collapsed and only first main important attributes are displayed.

Expand / Collapse Workspace control (3) hides/shows Activity stream and Case properties and expands/collapses Workspace. When workspace is expanded 'Activity stream & Properties' link is displayed in the right bottom corner of the page to open Activity stream and Case properties sections.

Below the Workspace Activity stream (4) is displayed. All activities such as comments and transitions through the workflows for every attribute, general comment, issue's reassignments are tracked in Activity stream.

Case properties (5) include information on the case (assignee, created date etc.)

User can navigate to 'Details 360' page by clicking 'Details 360' link.

Creating a Case

In order to create Manual cleansing case comment should be added or any attribute should be edited. After the case is created – Activity Stream and Case Properties sections gets refreshed and display relevant information. ID of the case is set in the bread crumbs as well.

The fields of the Case Properties section are configurable using the Administration portal.

Adding Comment on Attribute and the Whole Case

The User may comment on any data attribute. When the User rolls over the Original Data value or label the "Add comment" icon is displayed. When the User clicks on the icon the "Add comment" overlay is displayed.

The User may enter text and save a comment by clicking on appropriate "Save" icon. The comment will be then displayed in the Activity Stream and can be viewed when clicked on the "Add a comment" icon (comment will be displayed in the expandable area below the row).

Note: when comment is not yet saved – 'Add comment' icon is blue and comment box is yellow. When the comment is saved – icon and box both turn green.

Comments for attributes can be updated or deleted. In order to update already saved comment – user should click anywhere in the Comment box and comment text will be editable again. If a user wants to delete an inline comment, the user should click the 'X' icon.

Note: Every comment creates a ticket that is saved in database. They cannot be deleted manually; they can only be closed by workflow. So 'Delete' option only erases the value of the edit, but does not delete the ticket itself.

The User may optionally enter a global comment for the whole case by clicking "Comment" button. It will appear in the Activity stream.

Editing Attributes

Edit column by default is empty until the User edits value manually. The edit functionality is easily discovered when the User rolls over the editable fields – they are highlighted and tooltip ("Click to edit") is displayed.

When the cell is clicked the edit text field is displayed instead of the empty cell. The user can close the text fields without any changes made just by taking the focus out of the edited field.

When user clicks on "Save" icon for the edited field – Workflow actions drop-down appear.

Before letting the user save the changes – the edit gets validated. If validation fails – appropriate message is displayed and edit cannot be saved.

There is a possibility to prohibit editing of some fields. It can be done via Administration portal. When field cannot be edited, it is disabled and an appropriate hint is displayed.

A single edit may be reverted with "Revert Edit" option. Moreover, last made change may be undone.

These options are accessible from the context menu which appears when "x" icon is clicked. Also, value of the edit may be deleted, when 'Delete' option is selected.

Note: every edit creates a ticket that is saved in database. They cannot be deleted manually; they can only be closed by workflow. So 'Delete' option only erases the value of the edit, but does not delete the ticket itself.

Submitting Comments/Edits to the Workflow

The comments/edits are not sent to workflow until the User selects a necessary workflow action in a row and then click the "Submit" button. To select an action the User select a value from the drop down, then Submit button appears. To uncheck the action the User may select 'Select action' value (Submit button will disappear).

The user utilizes "Submit" buttons to make transitions one-by-one. Moreover, when the action is selected, the check box next to action is checked. It is utilized for bulk transitions. When workflow actions are selected for several edits "Submit Selected" button gets enabled and it can be used to submit several values at once. When actions are submitted – appropriate notification is displayed in the upper right corner of the screen.

All selected transitions that haven't been submitted yet can be reverted with "Reset All" button. The "Reset All" action doesn't apply to changes made to the values.

Fields in Edited column are displayed differently according to the workflow state they're in:

When the field is being Edited and not yet sent to the workflow:

When the edited field is sent to the workflow and is waiting for approval or rejection. It cannot be edited:

When Edited field is approved. It cannot be edited:

When the Edited field is rejected and goes back to the creator of the edit, so it can be edited by the creator.

After comment/edit is saved and page is refreshed – remediation icon appears for appropriate field(s). Pop up with short information appears when hovering on the icon.

Viewing Comments, Edits, or a Source Record

To view only attributes that require actions from the user, the user may use the "Hide Fields with no Activity" switch. When the switch is activated, only those attributes are displayed that:

  • Have manual edits.
  • Have comments.
  • Have issues (errors) from automated processes.

For convenience, the User may expand all the comments, errors and notifications for all attributes using "Expand all Comments/Errors" button.

Once clicked – buttons change their names: 'Hide fields with no Activity' -> 'Show All fields', 'Expand all Comments/Errors' -> 'Collapse all Comments/Errors', so a user can decide what they want to view.

A user can click the 'Show Source Record' button when they want to compare values of Instance and Source records. Once clicked, values of Source record appear to the right from values of Instance record and the button changes its name to 'Hide Source Record'. Values of Source record that differ from values of Instance record are highlighted in pink, and standardized values are highlighted in amber.

Note: values of Source record cannot be overridden. Only values of Instance record are included in workflow.

When user has some unsaved changes made on the page and tries to navigate outside of it – warning dialog will appear:

Created Manual Cleansing case is displayed on the Home page accordingly:

The row with open case is highlighted in amber color and appropriate icon is displayed.

Reassigning a Case

If a user wants to reassign the case to someone else, then the user should click the "Reassign" link in the Case Properties panel.

Note: 'Reassign' link is not available for closed cases.

If there are any unsaved changes on the page and users clicks 'Reassign' link – then warning dialog appears, so that case with unsaved changes could not be reassigned.

When there is no unsaved data on the page and the link is clicked – Reassign dialog appears. Functionality of it is similar to Assign dialog on Issues page.

A list of users is available in 'Assign to User' drop down menu. When user starts typing in the field – data is filtered dynamically to display only the relevant information.

If a user wants to assign some case(s) to themselves, the user can click the 'Assign to me' link:

Once the link is clicked, the currently logged in user is set in the field.

Note: When assigning case to the current user, the user is not redirected to the Issue search page Instead, the user stays on the same page and can work with the case.

If case cannot be assigned to the user selected from the list (for example, because selected user has no permissions to resolve cases, etc.) or entered data finds no matches in the database – appropriate validation message is displayed.

In case the selected user is valid, the "Assign" button is enabled and the case(s) can be reassigned.

After "Assign" button is clicked – user is redirected to Issues page because case is no longer their own.

Note: If you want an assignee to be notified about a reassign activity, then ensure that the Send email notification check box is selected.

Workflow Schema

The Workflow actions available for each edit are dependent on the User Role and the current status of the edit. In general they are described below:

When ticket is closed with help of Resolve Direct or Resolve -> Approve workflow actions – then Edited value overrides the Original one in appropriate Instance record.

If ticket is closed with help of any other workflow action – then override process does not start.

Workflow actions available per role are displayed below:

When the case is closed – records that were under cleansing remediation process are highlighted on the Home page with green color. New cleansing remediation process can be started for them.

User can close the whole case using 'Close Case' button. It appears only after all tickets are closed.

Note: Case can be closed only by assignee of the whole case.

If there are any unsaved changes on the page and users clicks 'Close case' button – then warning dialog appears, so that case with unsaved changes could not be closed.

After the case is closed – it is displayed in Display mode and cannot be edited or reassigned.