Data validation in-line editor

Data validation in-line editor

Data validation in-line editor

Errors or warnings relating to uploaded data can now be analyzed and corrected directly within the gather360 user interface.

Release 2.9 deploys a new in-line editor allowing users to correct errors or warnings from within the gather360 interface.

Following this feature deployment, users have two options to edit incorrect data:

  1. Edit inline on the gather360 application (recommended for a small number of corrections)

  2. Download a marked-up file to edit in Excel (recommended for large numbers of corrections)

New in-line editor

This new feature allows users to stay within the user interface to view and correct data errors and warnings, offering a more streamlined experience. The feature also tracks the changes the user has made to correct data issues, meaning the user has a better audit trail of how data was altered in order to pass the gather validation tests.

Watch our video run-through of this new functionality below:

In-line editor user guide

"Manage Errors" tab added to the Data Upload flow

The "Manage Errors" tab has been added to the standard data upload flow. After processing an uploaded file the user will be directed to the new tab to address any data errors that may have appeared.

Error summary report

After a file has finished processing, users will be presented with a summary of issues encountered. These will be categorized by issue type i.e.

  • Critical error (red) - A high severity error that must be resolved by changing the underlying data or the mapping before an output file can be generated

  • Warning (yellow) - A low severity error that can optionally be resolved. An output file will be generated when only warnings are present.

  • Correction (green) - A change made within the "Manage Errors" tab that has resolved a critical error or a warning

A further breakdown is shown by rule description to give the user an indication of what the error is and the type of correction that needs to be done.

The error summary report will be opened automatically when a file is processed. It can also be accessed at any stage by clicking on the "Summary" button.

Export to Excel option

If the user chooses to resolve errors using the standard option of downloading the error report in Excel format, they can do that by clicking on the "Export to Excel" button. This gives the option of downloading the raw source file and also the latest version of the validation error file.

Input row errors presented in a data grid

Alternatively, users can use the new in-line grid to correct rows that have shown at least one error or warning.

The grid is a direct replica of the data that has been uploaded, with 3 additional columns added for reference - "rowindex" (the row number from the uploaded file), versions (the version number of the data row, clickable to see previous versions) and "rule_messages" (a list of errors or warnings seen on the row). Note, the grid will only display those rows that have triggered a validation rule.

The grid is editable by double-clicking on a given cell. Changes are saved by clicking the "Save" button. The grid assumes a data type for each column based on the first row of data and prompts the user when saving to enter data to fit that assumed type. i.e. if a date value needs to be corrected (as below) and the field has been assumed to be of a date format, the user must compose the date in the correct format of "yyyy-mm-ddThh:mm:ssZ". If this formatting is too limiting for your specific scenario, then we would suggest using the old approach of editing and re-uploading the source file.

Colour-coded error indicators

To help assist the user in identifying different types of data issues, cells within the grid have been colour-coded.

  • red for critical errors,

  • yellow for warnings,

  • orange for updated cells that have not resolved an error,

  • and green for corrections.

When hovering over a cell with a data issue, the rule description that triggered the issue will be shown.

Filter/sort the grid

The grid can be filtered by clicking on the filter option on the right-hand side of any column. If it is a string or a date field, then filtering is a multi-select dropdown option. If it is a numeric field, then filtering has a standard numeric range filtering option. Filtering on "rule_messages" field allows you to narrow your set to rows affected by your chosen filter.

Press the "Unfilter" button to reset the grid to an unfiltered state. The tick boxes next to Unfilter allow you to filter rows based on errors, warnings or corrections.

View older versions of changed data rows

Any changes made to a row can be viewed in the versions field. Click on the version number to popup the row change-set ordered by version number descending.

Re-process the file after corrections

Once satisfied that all changes have been made, the file can be re-processed by clicking on the "Re-Process" button. This will regenerate the grid to update the new status of errors, warnings and corrections.

    • Related Articles

    • Map data from source to target

      A mapping enables you to define the columns to upload from your original file, and map them to the required fields. The gather360 tool also enables you to create expressions and formulas to transform the structure of data at the mapping stage. This ...
    • Upload data as a data consumer

      How data consumers can upload data This flow is primarily used by data consumers who do not require a supplier to upload data on their behalf. Usually, the consumer will already have access to the data that has been previously sent via email or SFTP ...
    • Data preparation time metric Data preparation time metric Learn more about the 'data preparation time' metric in your workspace KPI dashboard. The data preparation metric on your Workspace Dashboard ...
    • Manage data bottlenecks

      Manage data bottlenecks Help your data suppliers to understand data requirements so they can submit data on time. The gather360 KPI dashboard enables users to identify bottlenecks that are increasing your data preparation time and slowing down your ...
    • gather360 for data suppliers

      gather360 for data suppliers An introduction to the basics of gather360, for data suppliers that have been invited to join a workspace. Welcome to gather360 If you've recently received an invitation to join a workspace, start here to learn more about ...