SFDC Connector - Delta/Incremental



We use SFDC Datasets, and we've always pulled the entire dataset.  I've noticed their are options to do incremental/delta loads.  I am curious how to get started with these options.  Ideally i'd like to pull only new or changed records on a nightly basis as opposed to full data loads.


What i'm looking for is some guidance on how to properly configure this, i tried reviewing the documentation but i couldn't determine the proper approach to configure the selections on the SFDC Dataset Connector.


1. Load history

2. Begin loading only the changes nightly (which covers the past and the future)


Thank you!

Best Answer

  • jaredpk
    Accepted Answer

    Hi Nick -- I see where you are going with this.  One possible solution could be as follows:


    • Import only the last day's data based on system Mod Stamp
    • Compare the new data to the existing dataset in Domo.
    • Based on the lastest day's data and the dataset Unique identifier (opportunityID, AccountID etc.) recursively pull in the latest updates.


    I believe this could work with a little testing.  I'm leaving the dojo day session, but you can reach out to me with more specifics of the dataset if you would like.