Creating a dataset
To enable gather360 to prepare and request data for your organisation, you must define the target state for the data that you need. To do this, you must create a dataset.
A dataset specifies the target schema for how your data should be output. This includes a list of fields, and rules to define how your fields should be validated, transformed, and/or enriched.
Users or data suppliers can upload data into a dataset to validate and transform it to your desired state.
Setting up a dataset
Datasets are easy to set up and can be edited until you have performed the first upload of data into this dataset. Once your first upload has been completed, your dataset is locked, and cannot be edited without contacting support.
To set up a dataset, follow the instructions below:
Create a clear name and description for the data in your dataset
To start building your dataset, click on the " + Create Dataset" button from your work-space home page.
Enter a name and description for your dataset. This will tell your data suppliers, partners and colleagues more information about what data your dataset contains and the purpose of collecting that data.
Click "Save and Continue".
Add target fields to your Dataset
In the 'define target fields' window, click the " + Field" button to add a new target data field.
In the pop up that opens, specify a name and description for the new field.
This description will help your data suppliers and colleagues understand what information is required in each field, and help them map their data to yours if required.
The data type can be text, number, integer or date.
By specifying this type, gather360 will automatically validate that received data matches the expected type and will return a critical error if submitted data is not in the expected format.Specify whether the field is mandatory or not using the checkbox.
If a field is marked as mandatory, then blank or null data entries into the field will trigger a critical error.Specify whether the field is part of the primary key using the checkbox.
Click "Add Field" once completed and repeat this action for any additional fields that you require. You can do this in multiple stages if required.
Press "Save" to save all fields against the data store.
Add validation rules to the Data Store
Enter the 'Rules' window.
Select your rule type by clicking the '+ Add Validation Rule' button.
Specify the name of the rule and the target field it applies to in the popup, and click 'Build Rule' to enter the rule builder interface.
The name of the rule will be displayed to suppliers and used in your Audit ID to describe the error or warning that has occurred. The selected target field will be the field that indicates the error or warning to the data uploader.
Use the rule builder functions to apply validation rules to your data. When complete, click 'Save'.
Select the 'Critical Error' box to apply severe warnings.
For validation rules, the severity level can be one of the following:Critical Error - If the rule fails on a critical error then this will mark the data entry against the applied field in red. Data cannot be submitted until all critical errors are resolved.
Warning - If the rule fails on a warning then this will mark the data entry against the applied field in yellow. Warnings can be ignored and a valid mapped output will be created if only warnings exist in the validation file.
Click "Save" to add the validation rule.
Repeat the above steps for all validation rules that you wish to add, and then move to adding transformation rules.
Add transformation rules to the Data Store
Enter the 'Rules' window.
Select your rule type by clicking the '+ Add Transformation Rule' button.
Specify the name of the rule and the target field it applies to in the popup, and click 'Build Rule' to enter the rule builder interface.
The name of the rule will be displayed to suppliers and used in your Audit ID to describe the error or warning that has occurred. The selected target field will be the field that indicates the error or warning to the data uploader.
Use the rule builder functions to apply validation rules to your data.
Click "Save" in the rule builder and the popup window to add the transformation rule.
Repeat the above steps for all transformation rules that you wish to add, and then click "Save" in the Rules window.
You have now completed the setup for a dataset, and can begin uploading data.
Related Articles
What is a dataset?
An overview of the core elements within a dataset A gather360 dataset is a core entity that users can customise within the platform. It contains detailed settings and requirements for your specific data need. These settings define three core things ...Building Rules
An overview of rule-building on gather360. What are rules? The gather360 rules engine offers a range of ways to apply data governance and business rules to prepare and transform data into a report-ready format. In this article, you'll find ...Allowed file upload type
Upload .xlsx, .json, and .csv data files into gather360 enables users or data suppliers to submit data to a dataset. This can be completed manually by uploading a file, or via an automated data connection. For both upload methods, data must be ...Allowed file upload types
Upload .xlsx, .json, and .csv data files into gather360 enables users or data suppliers to submit data to a dataset. This can be completed manually by uploading a file, or via an automated data connection. For both upload methods, data must be ...Scheduled requests
gather360 enables users to set up a repeat cadence to automatically request data from suppliers. This functionality is called 'Scheduled Requests'. What is a scheduled request? A scheduled request is a recurring request for data to be uploaded from a ...