Dataflow: Ability to write R or Python code within a ETL-Dataflow
ETL dataflows are a simple to use pipleline model for transformation, however there are times when I'd like to include R or Python code as transformation so I can leverage an existing library, e.g., machine learning, text mining, etc.
My code would read a table from input, e.g., like this
df <- DomoR::fetch()
transform the data using R or Python code/libraries, then output the table to the pipleline, eg.
The ELT block would be a transient black box that reads from the input and write to the output. I imagine this could be done using a container that is quickly spun up to run the R or Python code.
In the longer term, I'd like to be able to create standard ETL blocks that I can drag and drop from the ETL palette. Adding this would make Domo competative with RapidMinder, https://en.wikipedia.org/wiki/RapidMiner