Average run time of a redshift data flow on large dataset
Hello, I wanted to get a general idea of your redshift run time. Any input is appreciated.
I have a data set with about million rows and 123 columns. However, it took 3.5 hours to simply have a subset of 1 column and almost 10 hours to run my data flow with some left joints. This dataset need to be refreshed everyday, so you can image how difficult it is for me to get anything done.
I am sure this data set is not a "Big Data" at all, curious how long does it take you to run a similar size of data.
Also, please give me any advise on how to speed up.
- 10.7K All Categories
- 13 Getting Started in the Community
- 37 Beastmode & Analytics
- 2.1K Data Platform & Data Science
- 59 Domo Everywhere
- 2.7K Charting
- 2.5K Ideas Exchange
- 1.3K Connectors
- 362 Workbench
- 303 Use Cases & Best Practices
- 500 APIs
- 118 Apps
- 48 News
- 753 Onboarding
- 1.2K 日本支部
- Private Company Board