Magic ETL dataflow not running because of no change in data

Reply
Highlighted
Major Blue Belt

Magic ETL dataflow not running because of no change in data

Hello,

 

So, I created a magic ETL dataflow that selects a row from a calendar where the shortdate=Current Date and I scheduled this dataflow to run at 6am daily.  The dataflow has not run once becuase it states the following:

 

Your DataFlow was last triggered on 4/3/2019 6:00 AM but there was no change in the data, so it didn't need to run. This DataFlow last ran on 3/23/2019 3:44 PM.

 

If I manually run this dataflow it will update and contain the correct record.  The data would change if the dataflow would run as scheduled.  What is happening and better yet how can I make this dataflow run on schedule?

 

Thank you,

Chris


-----------------
Chris

Accepted Solutions
Black Belt

Re: Magic ETL dataflow not running because of no change in data

That should work.  Essentially, Domo added a feature a while back that looks at the input data sets and if none of that data changes, then it won't run the job again.  

 

You need to have some kind of input data set (even if you don't end up using any of the fields) that will change each day.


______________________________________________________________________________________________
“There is a superhero in all of us, we just need the courage to put on the cape.” -Superman
______________________________________________________________________________________________

All Replies
Black Belt

Re: Magic ETL dataflow not running because of no change in data

If the data sets are not changing, is your expected output changing?  You might want to try bringing in a data set that changes each day.  (like one of the NOAA weather datasets)  That way you would have a change in the input and the data set would fire.


______________________________________________________________________________________________
“There is a superhero in all of us, we just need the courage to put on the cape.” -Superman
______________________________________________________________________________________________
Major Blue Belt

Re: Magic ETL dataflow not running because of no change in data

Typically the calendar table will not change.  If I understand you correctly this is why the dataflow does not fire.  Domo will not run at the scheduled time because the calendar table has not changed even though my output would be different.

 

Maybe I will create a workbench job that runs daily at 6am that outputs the current date and use that to kick off dataflow?


-----------------
Chris
Black Belt

Re: Magic ETL dataflow not running because of no change in data

That should work.  Essentially, Domo added a feature a while back that looks at the input data sets and if none of that data changes, then it won't run the job again.  

 

You need to have some kind of input data set (even if you don't end up using any of the fields) that will change each day.


______________________________________________________________________________________________
“There is a superhero in all of us, we just need the courage to put on the cape.” -Superman
______________________________________________________________________________________________
Announcements
Looking for the latest Community solutions? Please visit our accepted solutions board here!