Is there a reliable way to make Magic ETL intentionally fail?
I have an ETL that uses, among other inputs, a Webform that isn't always maintained perfectly. One recent problem involved two rows for the same identifier, with different text in the description column.
What I'd like to do is make the ETL fail if something like this happens. I tried doing a Group By on the identifier column, count the values, then filtered for counts greater than 1, and then tried to force an error by dividing by 0, but this results in "infinity". Similarly, SQRT() of a negative number produces "NaN", but no error.
I know there's potentially other approaches to this problem, such as creating an ETL that does validation across multiple datasets, then set up an alert based on certain outputs, but in the meantime, alerts won't prevent the ETL from producing incorrect results. I'd like the ETL to stop until the problem is fixed.
- 7.3K All Categories
- 13 Getting Started in the Community
- 141 Beastmode & Analytics
- 1.8K Data Platform & Data Science
- 54 Domo Everywhere
- 2K Charting
- 1K Ideas Exchange
- 903 Connectors
- 236 Workbench
- 342 APIs
- 77 Apps
- 19 Governance & Productivity
- 235 Use Cases & Best Practices
- 50 News
- 473 Onboarding
- 573 日本支部