Copy Beastmodes to New Dataset

Reply
Highlighted
Yellow Belt

Copy Beastmodes to New Dataset

I have a case where I have a dataset that has to large of a date range that is causing a query to run for a long period of time.  My solution is to create an archived dataset and a current dataset and then use a SQL join within DOMO to stack them.    This solution has worked well to create a dataset I want, however, I have many beastmodes on the original dataset and I want to move them to this new one created by the union.

 

My plan is to review multiple other datasets that are in this similar situation so that I can relieve resources on our servers for the queries.

 

Is there anyway to copy all beastmodes from one dataset into another?


Accepted Solutions
Highlighted
Major Red Belt

Re: Copy Beastmodes to New Dataset

@Benjamin ,

 

splitting data into Current and Archive might be a solution... but are you sure the problem will be resolved by splitting the data into two sets?  obviously i can't see your ETL, but it sounds like you're just moving your execution time bottleneck to another place... how are you building up your archive?  b/c as that dataset grows, ETL execution time is going to just increase, unless you're not using ETL tools.

 

AFAIk, unless you query APIs directly, there isn't a way to copy beast modes from one dataset to another (if the beast mode is saved to the dataset).  if you move a card from one dataset to another AND the beast modes referenced in the card are saved to the card, THEN they'll copy over assuming the column names are the same.

 

There are new tools like the Beast Mode Manager (a beta product, talk to your CSM) which will make managing Beast modes easier, but AFAIK in this current release, there isn't a way to copy from one dataset to another.

 

you could use the Domo Governance Beast Modes dataset to identify all the beast modes saved to a dataset, and then just basically create a checklist of every BM you need to recreate on the new dataset.

 

hope that helps!


Jae Wilson
Check out my Domo Training YouTube Channel

**Say "Thanks" by clicking the heart in the post that helped you.
**Please mark the post that solves your problem by clicking on "Accept as Solution"

View solution in original post


All Replies
Highlighted
Major Red Belt

Re: Copy Beastmodes to New Dataset

@Benjamin ,

 

splitting data into Current and Archive might be a solution... but are you sure the problem will be resolved by splitting the data into two sets?  obviously i can't see your ETL, but it sounds like you're just moving your execution time bottleneck to another place... how are you building up your archive?  b/c as that dataset grows, ETL execution time is going to just increase, unless you're not using ETL tools.

 

AFAIk, unless you query APIs directly, there isn't a way to copy beast modes from one dataset to another (if the beast mode is saved to the dataset).  if you move a card from one dataset to another AND the beast modes referenced in the card are saved to the card, THEN they'll copy over assuming the column names are the same.

 

There are new tools like the Beast Mode Manager (a beta product, talk to your CSM) which will make managing Beast modes easier, but AFAIK in this current release, there isn't a way to copy from one dataset to another.

 

you could use the Domo Governance Beast Modes dataset to identify all the beast modes saved to a dataset, and then just basically create a checklist of every BM you need to recreate on the new dataset.

 

hope that helps!


Jae Wilson
Check out my Domo Training YouTube Channel

**Say "Thanks" by clicking the heart in the post that helped you.
**Please mark the post that solves your problem by clicking on "Accept as Solution"

View solution in original post

Highlighted
Yellow Belt

Re: Copy Beastmodes to New Dataset

I appreciate your response.

 

Let me give you a little more background and see your thoughts.

 

Originally I had this dataset setup to pull from 2017 through today and it would take approx 15 minutes to run.  My end goal was to shorten the amount of time it would take to update this dataset.  I am pulling this data into DOMO through workbench running a SQL query against our DB with an ODBC connection.

 

My approach was to create an archive dataset and a current dataset.  The thought is that I can make the archive run from 2017 to current year and current dataset run 2020 only.  I could only run the current dataset and append/join to the archive.

 

In testing the archive dataset takes about 8 minutes to run and the current about 1.5 minutes.  

 

I don't know the best method of stacking these in DOMO so I tried two.  The first I tried was through dataflows SQL and used a UNION for the two datasets.  This takes approx 25 minutes to run so it was not a better solutoin.

 

My second try was through dataflows magic ETL and I used the append function.  Surprisingly this only took 8 minutes.  So now I've reduced the overall update time from 15 minutes to 8.   

 

I'm not sure what additional methods I should try that may be faster?

Highlighted
Yellow Belt

Re: Copy Beastmodes to New Dataset

I just tried one additional test and this time I used the Domo Blend function with the Add Rows feature and that appeared to be almost instantaneous.  

 

If there is no downside of this being a datafusion blend then this appears to be the best method?  It would cut down my update time from 15 minutes to approx 2 minutes.

Highlighted
Brown Belt

Re: Copy Beastmodes to New Dataset

@Benjamin - DataFusions are a lot quicker than an ETL. I try to use them first if at all possible. The one current downside is that you can't use the fustion dataset as an input into a future ETL (Magic or MySQL) but I believe there might be a beta which will allow just that (along with DataSet Views / another beta).



**Was this post helpful? Click the heart icon**

**Did this solve your problem? Accept it as a solution!**
Announcements
What is your Dojo rank and what badges have you earned? How am I doing? Get a refresher on how to up your rank and collect badges in the Dojo here.