The gather360 rules engine offers a range of ways to apply data governance and business rules to prepare and transform data into a report-ready format.
In this article, you'll find information on:
1. Rule Types
There are several different types of rules available:
These rules enable data submissions to be filtered before upload to eliminate unnecessary rows.
Mapping rules make it easy to match the fields of a data upload to the target state in a gather360 dataset. gather360 enables the creation of complex mappings for multiple fields, featuring the option to use mathematical expressions and concatenation options to map data to the target state.
In addition, the gather360 interface enables the creation of multiple pre-configured mappings per data source.
Validation rules govern and cleanse submitted data. The system checks data uploads against requirements and prompts data suppliers to resolve errors before submitting data to ensure that all submitted data meets quality requirements.
Validation rules can vary in complexity and are custom configurable within the gather360 interface. These rules always return a 'True' or 'False' status. True means the data has been validated and meets requirements. False means the data has been validated but does not meet requirements.
There are three categories of Validation rules:
Basic: When adding fields in gather360, we specify the format (i.e. string, integer, number or date) and whether it is mandatory or optional. The system automatically generates rules to test all submitted data against this field criteria.
Warning: Data that returns a 'False' status will be flagged to the uploader but won't prevent submission. The error can be optionally resolved and re-tested. If the error is not resolved, the data submission will be flagged and made available for analysis in the data layer.
Critical Error: Data that returns a 'False' status will be prevented from submission. The error must be resolved and the validation test repeated before data will be accepted.
Transformation rules convert data fields from one format to another. Transformation rules consume input data or available enrichment data to create a new data point based on configured requirements.
gather360's flexible rule engine can be easily configured to support most business data rules.
The system uses 'logical operators' to enable complex multi-level rule building in a formula-like environment. The system utilises mathematical expressions and boolean logic to convert business logic rules into rule functions.
Where there is an ongoing need for complex rules to be utilised across the business, gather360 enables the creation of customised functional operators that can be saved in a simplified format for re-use.
Rules can be triggered from or include any data point that has been uploaded into the gather360 system and can incorporate data from custom reference tables or enrichment data. The scope of data that can be used in rule building is shown below:
A field - The value stored in the field.
A row - The value stored in the field with one or more values within that row
A data file - The values stored in a field/row with any other value or rows within the data file
Other data sets - The value stored in field/row with any other value stored in other datasets previously uploaded
Other reference sources - The value stored in a field/row with any other reference data store made available to the platform for lookup and processing.
Please see a selection of rule examples below:
Lookup rules: checking fields against a defined reference table list (e.g. currency, country code, ZIP code)
Conditional rules: specifying an allowed range for values (e.g. date range, price range, time range)
Chronological /Combinational rules: creating a required combination or sequence that data must follow (e.g. Column A must be within 30 days of Column B)
Computational rules: mathematical expressions (e.g. sum of orders must equal total order amount)
Transformation rules: formulae to convert the format of data from A to B (e.g. exchange USD to GBP or split DD-MM-YYYY to DD-MM)
The gather360 rules engine is very flexible, so there are very few limitations on rule building. Where complex additional functional rules are required, the g360 technical support team can provide these via our helpdesk.