Field Dependencies Documentation
There have been several requests for ERD style maps of our datasets, however I'd like to see these dependency maps taken to the field level.
If we have a field in a dataset that is result of a series of dataflows we need to know how its derived - what transforms have acted on that field from the raw data to the final dataset used to build cards. This is not just for the general knowlege but to see how changing one dataflow may impact the final dataset.
Breaking down a dataflow into multiple steps (some MySQL, some magic ETL) as opposed to a wall of SQL is great for creating the flows and visualizing the relationships, however it creates a lack of transparency when determining cascading dependencies from one flow to the next.
We've resorted to using a seperate documentation tool to achieve this but it is work intensive, and is in constant danger of being out of date.