Data Flow help
I am humbly reaching out for some help.
I have a vendor sending a snapshot of a database table every day and not sending me diffs. I am trying to determine the diffs. The connector type is SFTP CSV. The file they send daily is overwritten daily. I currently have the data connector appending the data from the previous day to the new file they sent over that day.
Is this the correct approach? Should I have the connector setup as Append vs Replace or vice versa?
What success should look like as an output dataset that appends updated records? Below, you will find an image of 6 records that are the same as an individual in the file.
As you can see, it's the same row the only difference is the row_id which is really when the file was run and appended per the connector. So for example, imagine that these 6 rows are John Smith, what we care to see is the original John Smith record and only any new/updated changes. So if there were three changes I would only see three new records.
Thank you so much in advance. We are happy to answer any questions.
- 7.3K All Categories
- 13 Getting Started in the Community
- 141 Beastmode & Analytics
- 1.8K Data Platform & Data Science
- 53 Domo Everywhere
- 2K Charting
- 999 Ideas Exchange
- 902 Connectors
- 236 Workbench
- 342 APIs
- 77 Apps
- 19 Governance & Productivity
- 234 Use Cases & Best Practices
- 50 News
- 473 Onboarding
- 572 日本支部