Fusion, how many rows is too many?

Reply
Highlighted
White Belt

Fusion, how many rows is too many?

I have been watching @jaeW_at_Onyx on Youtube and am seriously excited by his data fusion methods. I would like to mimic his 
how to build a Trial Balance, Income Statement and Balance Sheet with no ETL but have 28 million rows in my GL. Would it be more efficient to split said 28 million into yearly chunks,  first using a row fusion and then add metric table in a column fusion or just have them in one huge dataset. I am currently using workbench to import and append recent transactions every 15 mins.

Thanks, David
Tags (2)
Highlighted
Brown Belt

Re: Fusion, how many rows is too many?

@grimmers - Fusions should be able to easily handle 23M records. It should handle up to hunders of millions of rows just fine. Plus with the benefit of fusions updating near instantaneously they're a really great tool for the large data sets.



**Was this post helpful? Click the heart icon**

**Did this solve your problem? Accept it as a solution!**
Highlighted
White Belt

Re: Fusion, how many rows is too many?

@GrantSmith thanks for the reply. I am curious to know how Domo will handle this much data in a fusion with a cross join to a dimension table. I will be giving it a try today so fingers crossed!

Thanks, David
Highlighted
Major Red Belt

Re: Fusion, how many rows is too many?

you have to be careful if you're adding new data every 15 minutes.

 

When you add new data, Domo has to reindex the data in Adrenaline.  If Adrenaline re-indexes the Fusions, I'm pretty confident it clears your cache.  If you clear your cache, all your pages and cards will never benefit from a warm cache.

 

It should be fine as long as your cards are primarily dealing with low cardinality (few distinct values) data.  If you're pulling in long text strings (descriptions of transactions) that can impact performance as Adrenaline compresses your data.

 

If you're worried about performance, cut long free text columns.  

 

I don't think I would split up data into years.

 

Let us know how it goes!

 


Jae Wilson
Check out my Domo Training YouTube Channel

**Say "Thanks" by clicking the heart in the post that helped you.
**Please mark the post that solves your problem by clicking on "Accept as Solution"
Highlighted
White Belt

Re: Fusion, how many rows is too many?

OK, after some testing with my 28 million rows and adding a cross join to a dimension table the blow out table went  to 340 million rows with no visible performance issues. even with 15 minute updates of the transaction table there have been no problems. I am now a Fusion convert!!

Thanks to @GrantSmith , @jaeW_at_Onyx  for their input

 

 

 

Thanks, David
Highlighted
Major Red Belt

Re: Fusion, how many rows is too many?

@grimmers  that's wicked!  glad to hear.  If your dashboard does slow down, consider dropping columns in your Fusion.  

 

If any of the responses from @GrantSmith  or me helped, please feel free to mark them as a solution!  Thanks!


Jae Wilson
Check out my Domo Training YouTube Channel

**Say "Thanks" by clicking the heart in the post that helped you.
**Please mark the post that solves your problem by clicking on "Accept as Solution"
Announcements
Stand out in the Dojo: Choose your own recognizable username and add a unique profile photo in your profile settings.