Trying to filter out rows, where a certain column has NULL values
Hi,
I have a dataset of almost 2 billion rows. I need to filter out rows, where a certain column has NULL values.
What is the best and fastest way to do it?
I tried creating a DataFlow using the mySQL option, but the flow has been running for more than 24 hours now.
Could any of you please help in understanding how this process can be made to run faster?
Comments
-
With large datasets, I normally have more luck (where speed is concerned) using the ETL tool.
You could accomplish what you're looking for using 3 widgets, Input Dataset into Filter Rows into Output Dataset.
On the filter rows, just select the row you want to filter and there is an option for IS NOT NULL.
That should hopefully be all you need. If not, just let me know.
Sincerely,
Valiant
**Please mark "Accept as Solution" if this post solves your problem
**Say "Thanks" by clicking the "heart" in the post that helped you.2 -
Are you needing to filter those rows out completely, or do you want to improve your filtering ability in your visualizations? Is this a text field or numeric?
If you want to keep the rows, but need the null values filled in, you can use the mapper. This works for both text and numeric fields.
If you are simply needing to remove them efficiently, @Valiant 's solution is perfect.
DataMaven
Breaking Down Silos - Building Bridges
Check out my video!
**Say "Thanks" by clicking the heart in the post that helped you.
**Please mark the post that solves your problem by clicking on "Accept as Solution"1
Categories
- 10.6K All Categories
- APAC User Group
- 12 Welcome
- 36 Domo News
- 9.6K Using Domo
- 1.9K Dataflows
- 2.4K Card Building
- 2.2K Ideas Exchange
- 1.2K Connectors
- 338 Workbench
- 250 Domo Best Practices
- 11 Domo Certification
- 460 Domo Developer
- 47 Domo Everywhere
- 100 Apps
- 703 New to Domo
- 84 Dojo
- Domopalooza
- 1.1K 日本支部
- 4 道場-日本支部へようこそ
- 22 お知らせ
- 62 Kowaza
- 295 仲間に相談
- 649 ひらめき共有