Append a dataset without using recursive design

Is there any other way to append to a dataset without using a recursive design in Magic ETL?


Best Answer

  • Jessica
    Jessica ⚪️
    Answer ✓

    Thanks I found the append in DOMO Workbench. The basic csv connector from Workbench does not have a setting on the DOMO application side.

    It's on the Workbench side.

    Thanks again


  • MarkSnodgrass
    MarkSnodgrass Portland, Oregon 🥷


    You will probably need to provide more specifics as to what fields are in your dataset to work with, but generally I would say yes you can do this without a recursive.

    In the ETL, you can split off the IDs and dates in your dataset and then group by max date. Then join it back to your main dataset and join on the date and id. This would give you the latest of each your unique rows.

    Hope this helps.

    **Check out my Domo Tips & Tricks Videos

    **Make sure to <3 any users posts that helped you.
    **Please mark as accepted the ones who solved your issue.
  • GrantSmith
    GrantSmith Indiana 🥷


    It depends if you're wanting to ignore duplicate records or not and how you're pulling in your data. If you're using Workbench you could utilize and UPSERT to overwrite existing records or an APPEND to just tack on the result set to the existing domo dataset.

    Another option would be to utilize a dataset copy connector to take snapshots and set it to append but it could possibly have duplicate rows.

    Most connectors also support APPEND as the update method.

    **Was this post helpful? Click Agree or Like below**
    **Did this solve your problem? Accept it as a solution!**
  • jaeW_at_Onyx
    jaeW_at_Onyx Budapest / Portland, OR 🔴

    Without hacking Domo, you can only APPEND or UPSERT data during ingest (not after ETLs have been constructed). That said, if you get crafty you can alter a dataflow to APPEND instead of REPLACE (but not UPSERT)

    In this video i got the dataflow to APPEND but was unable to get it to UPSERT as desired.

    Jae Wilson
    Check out my 🎥 Domo Training YouTube Channel 👨‍💻

    **Say "Thanks" by clicking the ❤️ in the post that helped you.
    **Please mark the post that solves your problem by clicking on "Accept as Solution"