Joining based on conditions in Magic ETL

Can we make joins based on conditions in Magic ETL? e.g. I want to replicate the below in a join in magic ETL. Can I do it?

 

SELECT

dataset_1.*

dataset_2.*

FROM dataset_1

LEFT JOIN `dataset_2` on
(dataset_2.id` IS NULL
AND `dataset_1`.`Day` >= COALESCE(dataset_2.date`,dataset_1.date`) )

Comments

  • GrantSmith
    GrantSmith Indiana 🔴

    Hi @user022825 

    Currently you can't exactly do conditional joins with Magic ETL. You'd need to create a single (or multiple) columns you can join on and then do your filtering / conditional join logic after the join has been done.

     

    Simply put there isn't an easy way of doing this type of join with Magic ETL. There are possible other ways but it would involve greatly increasing your dataset size thus increasing your processing speed.

     

  • jaeW_at_Onyx
    jaeW_at_Onyx Budapest / Portland, OR 🟤

    if the datasets are of reasonable size you can do a cross apply on dataset A to dataset B (join where 1 = 1) then use a FILTER tile to remove rows that don't meet your JOIN criteria. 

     

    It's not great, but it gets the job done.