A better approach to validating Datasets
Domo reports can only be trusted if data sets have been validated, however Domo does not appear to provide much support for validation and work needs to be manaully validate.
1) Domo provides a rapid building environment (Magic) for joining and transforming data, however the risk is the new data set does not accurately reflect the inputs.
2) There are existing reports, generated outside of domo, that we need to validate against.
3) Input data sets can introduce problems long after the transforms have been created.
Typical best practice is to create a few cards and sumos then manually validate the output data set agaisnt the inputs or the reports.
When doing data transformation in Python I typically wrote validation tests to automate this and ongoinly validate my data.
Create a validation tool that regularly checks a data set based on a set of rules/test such as:
- Type: Column is a Number or a Date etc.
- Value: Column is in [or not in] a range (numeric, date) or list (text), eg, 'Value' > 0 or Value NOT NULL.
- Relationship: Column value is in a related table (FK/PK relationship)
- Aggregate: An Aggregate in this table (or filtered selection) matches an aggrigate in another.
- Lenght: The lengh of a table (or filtered selection) matches some other table.
The validations whould be associated with the data set and probably based on SQL
There are probably more but, this woudl be a great start for checkign an output table against the inputs or an existing report table.
- 11.3K All Categories
- 6 Private Company Board
- 2 APAC User Group
- 12 Welcome
- 45 Domo News
- 10.2K Using Domo
- 2K Dataflows
- 2.6K Card Building
- 2.3K Ideas Exchange
- 1.3K Connectors
- 354 Workbench
- 271 Domo Best Practices
- 14 Domo Certification
- 483 Domo Developer
- 58 Domo Everywhere
- 113 Apps
- 734 New to Domo
- 86 Dojo
- 1.1K 日本支部
- 4 道場-日本支部へようこそ
- 31 お知らせ
- 65 Kowaza
- 305 仲間に相談
- 667 ひらめき共有