Mutiple Zip files files in S3 Bucket
I am having 2 problems with the S3 Connectors (Advanced and Standard). I am struggling to find which one is better suited to handle multiple files.
Here are the details:
Our vendor is posting a new Zip file each day to an S3 Bucket. The subfiles within the new daily Zip all have the prefix change based on a physical location identifier. For example,
where 1345 and 1298 are the location identifers
Using the advanced connector, I am only able to pull one json file at a time I include the prefix (i.e. 1345) in the SUBFILE field, however, it will only extract that first json file. I need to pull all the json files within that Zip. Would re-naming the files to something with a generic name file the issue (i.e. Daily File_1345_08_14_20.json) where i use the Daily_File in the subfile field?
The second question is how to append the new file(s) each day as we are not able to enter a variable. I see the regular S3 Connector will grab most recently modified, however, the Zip option requires that you discover the zipped files using a drop down.
Any hints or suggestions would be amazing!!! Thanks
- 10.7K All Categories
- 13 Getting Started in the Community
- 41 Beastmode & Analytics
- 2.1K Data Platform & Data Science
- 59 Domo Everywhere
- 2.7K Charting
- 2.5K Ideas Exchange
- 1.3K Connectors
- 362 Workbench
- 303 Use Cases & Best Practices
- 503 APIs
- 120 Apps
- 48 News
- 753 Onboarding
- 1.2K 日本支部
- Private Company Board