How to delete rows from an input dataset and replace the dataset
I was trying to use SQL transformations, but I cannot make the output dataset the same as the input dataset. Could someone help?
Comments
-
The solution is a partial recurive flow but in your case, you aren't maintaining history rather, delete/replacing the input with the output.
I've created a flow to demonstrate how you can do this using simple SQL flows adding a constant that will distinguish which rows are obsolete vs. not.
Let me know if this helps.
Test Data
Include a flag/constant that will help you distinguish which rows are obsolete
Choose the Custom SQL
Delete rows which are Obsolete
use this output as your new input
0 -
Thank you for the detailed explanation. I did not know about the DELETE command, and that was very helpful.
While this is a significant improvement from what I thought I could do, there's still this issue that remains:
The new data that comes in is never obsolete. Its only after a certain time that it becomes obsolete. That means that I cannot instantly do operations on incoming data, rather I have to store it in a dataset, and keep running delete statements on that to remove unwanted data.
So while my end output in purged, there is this huge dataset in the middle that I still have to maintain.
Any idea on how I can reduce the size of the middle dataset as well?
0
Categories
- 10.8K All Categories
- 3 Private Company Board
- 1 APAC User Group
- 12 Welcome
- 38 Domo News
- 9.7K Using Domo
- 1.9K Dataflows
- 2.5K Card Building
- 2.2K Ideas Exchange
- 1.2K Connectors
- 343 Workbench
- 258 Domo Best Practices
- 11 Domo Certification
- 464 Domo Developer
- 50 Domo Everywhere
- 105 Apps
- 714 New to Domo
- 85 Dojo
- Domopalooza
- 1.1K 日本支部
- 4 道場-日本支部へようこそ
- 26 お知らせ
- 64 Kowaza
- 299 仲間に相談
- 653 ひらめき共有