Timestamp Runs Recursive Dataflow Magic 2.0

mbelmont
mbelmont 🟡
in Data Platform & Data Science

Hello all!

I feel like I have lost it but I am looking for a simple way to timestamp each run in a recursive dataflow. The output dataset is simply appending on top of itself with no exclusions thereby producing a new set of rows based on the latest data available in the source dataset. When I try and timestamp I am setting the value for all rows and not just the most recent run. Anyone met with this challenge before?

Tagged:

Answers

  • GrantSmith
    GrantSmith Indiana 🥷

    You should add the timestamp to the data that changed / added and not the entire dataset. Do you have a screenshot of your Magic ETL for reference?

    **Was this post helpful? Click Agree or Like below**
    **Did this solve your problem? Accept it as a solution!**

Categories