Magic ETL dataflow not running because of no change in data

cwolman
cwolman 🟠
in Dataflows

Hello,

 

So, I created a magic ETL dataflow that selects a row from a calendar where the shortdate=Current Date and I scheduled this dataflow to run at 6am daily.  The dataflow has not run once becuase it states the following:

 

Your DataFlow was last triggered on 4/3/2019 6:00 AM but there was no change in the data, so it didn't need to run. This DataFlow last ran on 3/23/2019 3:44 PM.

 

If I manually run this dataflow it will update and contain the correct record.  The data would change if the dataflow would run as scheduled.  What is happening and better yet how can I make this dataflow run on schedule?

 

Thank you,

Chris


-----------------
Chris

Best Answer

  • ST_-Superman-_
    ST_-Superman-_ 🟤
    Answer ✓

    That should work.  Essentially, Domo added a feature a while back that looks at the input data sets and if none of that data changes, then it won't run the job again.  

     

    You need to have some kind of input data set (even if you don't end up using any of the fields) that will change each day.


    ______________________________________________________________________________________________
    “There is a superhero in all of us, we just need the courage to put on the cape.” -Superman
    ______________________________________________________________________________________________

Answers

  • ST_-Superman-_
    ST_-Superman-_ 🟤

    If the data sets are not changing, is your expected output changing?  You might want to try bringing in a data set that changes each day.  (like one of the NOAA weather datasets)  That way you would have a change in the input and the data set would fire.


    ______________________________________________________________________________________________
    “There is a superhero in all of us, we just need the courage to put on the cape.” -Superman
    ______________________________________________________________________________________________
  • cwolman
    cwolman 🟠

    Typically the calendar table will not change.  If I understand you correctly this is why the dataflow does not fire.  Domo will not run at the scheduled time because the calendar table has not changed even though my output would be different.

     

    Maybe I will create a workbench job that runs daily at 6am that outputs the current date and use that to kick off dataflow?


    -----------------
    Chris
  • rbaeza
    rbaeza ⚪️

    I would like to add something in case anyone else in 2021 is having this issue.

    My issue is that the input DaraSet I have is running after this job that is "not running". I needed to make sure the input DataSet ran first on the scheduler, and this fixed my issue!

Categories