This site uses different types of cookies, including analytics and functional cookies (its own and from other sites). To change your cookie settings or find out more, click here. If you continue browsing our website, you accept these cookies.
Upgrading Server to version 22.1? We’ve enabled the faster AMP engine by default. Read about how these system settings changes might affect you.
We’re aware of an intermittent issue with our My Alteryx login and are actively working to have the issue corrected. If you run into an error when logging in, please try clearing all cookies or accessing the community on a different browser. Thank you for your patience!
Connect currently provides lineage for a datasource at the table level but we would like to be able to have this at a more granular level i.e. the column level so if a specific column changes in the table (e.g. data type change), we are notified of the change and able to identify quickly assets that will be affected.
we are currently investigating options how to achieve this goal. It is clear that we cannot get the columns using the current approach, i.e. parsing the workflow definition, but need to get on a lower level.
This will probably have a significant impact on the whole lineage-making process and the results can be expected in rather long- than short-term.
This may also need @AdamR and his team to contribute - since this would be much easier if the engine had a unique field ID internally (similar to the way that compilers have a symbolic variable table that gives a unique memory address to each declared variable)
Field traceability, if done at the engine level, would also allow for optimisation hints like "you've never used this field in an output - may be worth eliminating early in the flow" - again, similar to the hints you get in code IDEs.
I definitely agree with you. Unfortunately this improvement is not much about Connect being able to show such information as about retrieving the information. We are looking for ways how to get it straight from engine, but I cannot promise anything at this point.
I got the same request from my business partners for column level lineage. For Connect to evolve as a competing data lineage, this is a critical feature to include.
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.