Combine Data Issues
I have 3 datasets called FYC Historical, FYC Details-Current, and FYC Summary. I have combined these with our VUE Production Clean Dataset and called the dataflow VUE FYC Combine (takes almost an hour to run).
I have then taken this output dataset, called VUE Historical and Current Test, and placed it into a copy of an overall dataflow called FYC TEST of Bridge | Course and Program Enrollment Status, which produces my 2 main datasets (FYC TEST Bridge | Course Enrollment Status and FYC TEST Bridge | Program Enrollment Status).
I have tried to run this dataflow, and it has gone longer than a day and ended up canceling with over 400 million rows and almost a terabyte of data. What am I doing wrong? What can I do to correct this to get it to run faster?