Documentation of Workflows by Alteryx

[caltang]

I am aware that an Auto-Documenter tool is available in the Gallery, but that has not been maintained since 2020. 

It would be great if Alteryx could have that as an added feature to the Designer as an option for end-users to utilize. 

The breakdown of it can be done via XML parsing as such: 

<Nodes>: Configuration of tools

<Connections>: The tools used

<Properties>: Workflow properties

Right now, the current workaround is for users to export their XML, and the internal Alteryx development team has to build another workflow that reads the XML accordingly + parses it to fit what is needed. 

It would be better for Alteryx to build something more robust, and perhaps even include some elements of AiDIN which they are promoting now.

[Rags1982]

I like this idea. 

This happens to be one of my jobs within our development team. I've built a workflow that parses out all the XML to give tool details etc, and to try and make some sense of the workflow. 

We even went one step further and tried to follow a field from input to output of the workflow, then using a directory of workflows, followed that field through. This helps with our internal governance and allows us to track which workflows/reports are affected by a field being changed earlier on in the process. The difficulty we had was down to the connections. As the connection nodes aren't necessarily in the right tool order, we had to create an iterative macro that will create the joins, just so I could follow a field through. Gets a bit slow with workflows with over 500 tools!

[caltang]

Thanks for sharing @Rags1982! Appreciate your support. 

Just curious, do you mind sharing more information about your macro? I’m thinking of putting something together to be appended to this post. 

Hope it gives ideas to the Alteryx engineering team as well.

[RemcoV]

This sounds like a very good suggestion, I would love to see it be implemented.

@Rags1982, your workflow sounds very nice. Would you mind sharing it on here/elsewhere on the community? 

[Rags1982]

Hi @caltang , @RemcoV ,

Thank you. 

Unfortunately I can't share the workflow and/or macro as it contains sensitive information. 

It is fairly simple though tbh. With the macro, all you need is an input containing the connections of a workflow. These can be parsed from the XML of a workflow under the 'Connections' Node. Each connection will have an 'input' and an 'output' tool ID. The iterative macro will read these and will try to join the output of record one to the input of another record. It will do this as many times as needed. 

The output of the macro then gives you all the connections of tools in that workflow in order. 

Example: 

I made the parent workflow faster by only using the longest possible number of connections that had both an input and an output (Hence the length field). This gave the longest stream of tools with the best possible chance to track a field.

I then joined the xml of each tool ID, from the parent workflow, back to the Tool ID in the identified stream. This gives the xml of each tool in a workflow in tool order. Then, by parsing that xml, you can obtain field names and renames. You can then use this to track a field from the start of a workflow, through all the tools, to the output. 

Now, the xml of an output tool doesn't contain field names, usually, but if you are outputting to sql as we do, you can write some sql script to pull all field names/columns in a table. Joining these field/column names to the output tool based on the table that the output is writing to will give you all needed info.

If you then join multiple workflows together, you can track a field through all the tools of all workflows. 

This does seem rather long winded, and I had to make things up as I went as I don't think this has been done before. Not that I could find online anyway. But, I can now track a field from one workflow to another, being able to track it through each tool and see what's happening to that field and data. 

Happy to explain more if specific points don't make sense, as this reply is getting quite big. Haha

[caltang]

Whilst waiting for an official response from Alteryx, I actually built a bare bones workflow that reads the XML. 

Mine just involves the descriptive stats for now - who wrote the workflow, the tools used, a simple count of tools and an export of the image. 

I use the spacing to determine Nodes and ChildNodes to know when they start and when they stop, so the contents can also be read. 

[Rags1982]

@caltang I've done so much work in our team with Alteryx XML. 

Aside from the above, I've built another app that will take the XML of a workflow, change a few things, add a few tools with fixed details in, rename it and save it elsewhere as another Alteryx workflow. This is basically for our governance. We have a Development, UAT and Production environment and workflows must be 'promoted' using this app in order to be placed into Production. 

I even linked it with another app that 'approves' a UAT workflow. A manager goes and approves the new workflow, then it can be promoted. Without approval you cant promote it. 

It will also write all the xml details like tools, write times, authors, workflow IDs, inputs/outputs etc to a sql table when you promote a workflow. 

You mentioned spacing? One tip I can give is to read the Alteryx file as a CSV with \0 delimiters rather than XML. It means that it will put the XML in an easier to read format when trying to to read tool data. You can always use a summarize tool, concatenating by \n to bring all the lines back together in order to get connections/property nodes

[caltang]

@Rags1982 mad respect man! You could package some of them with dummy data and submit it to the Gallery for the community to benefit from if you'd like. 

Also, thanks for the tips & ideas! I am hoping to build something of that sort as well. Would it be okay if we discuss off-site? 

Thanks!

[Rags1982]

Thanks @caltang 

Happy to give help when I can. 

[rohit782192]

Its a Great idea if this can be build

[caltang]

For anyone keen, I've got a crowdsourcing workflow going on here: https://community.alteryx.com/t5/Alteryx-Designer-Desktop-Discussions/Auto-Documentation-Maveryx-Cro...

Whilst we wait on Alteryx to respond to this, I think it'll be great for us to recognize the AI Summary tool they released, though it only tells what the tool is doing and not the whole bit. The config of each tool is important for audit purposes as well.