Magic ETL - Change column types and merge files
Hi, I am merging 7 data sets into one. I would like to change the column types on all 7 of the source files before merging into the output data set.
2 columns in each file are "text" columns, but I wish to change them to date type.
Some of the data contains "n/a" as a string. When I ran a simple Set Column Type ETL on this field, the "n/a" prevented the conversion.
I am applying a value mapper to both text columns containing the date (but with column type "text") to convert the "n/a" string to an arbitrary date (e.g. 1970-01-01).
Then I am applying the Set Column Type transformation to change those columns to column type date. After that I run an append rows transformation to complete the ETL and populate the Output file. See below a snippet of the Magic ETL.
When I run this, the ETL creates 2x the number of rows vs. the sum of the original 7 datasets.
Any ideas why this is and how I can achieve my goals (replace column types for 2 columns and combine all 7 sets)?
- 10.7K All Categories
- 13 Getting Started in the Community
- 39 Beastmode & Analytics
- 2.1K Data Platform & Data Science
- 59 Domo Everywhere
- 2.7K Charting
- 2.5K Ideas Exchange
- 1.3K Connectors
- 362 Workbench
- 303 Use Cases & Best Practices
- 503 APIs
- 120 Apps
- 48 News
- 753 Onboarding
- 1.2K 日本支部
- Private Company Board