Best practice - Recording Workflow hierarchy

[DataBlender]

Hi all

Bit of a general question, but I was wondering whether anybody had any examples of best practice that they could share with regards to documentation of workflows and how they hang together.

With different people working on different pieces of analysis, and to avoid a single workflow becoming too unwieldly, we find that we end up with multiple workflows which may or may not depend on the output of another workflow.

If original data or an intermediate workflow changes, then it is currently down to the user to remember to flush these changes through any dependent workflows. Is there a better way than just manually recording the workflow structure? 

Thanks

[KaneG · Accepted solution]

Hi @DataBlender,

This can be done by reading the XML of the modules. You can read in a workflow at Child:Node and then parse looking for Input & Output tools.

Attached is a VERY rough sample module that I give to customers and partners with the purpose that they can go and customise it the way they would like.

Normal changes that clients will make are:

Directory Input and dynamic input instead of multiple file inputs

Remove Text Input container as it errors if there is no text input tools

Re-organise the output - even using reporting tools

Add comment tools to the information that is parsed

Put in support for nested containers as currently anything inside a nested container will be discarded.

I do re-iterate that it is very rough, and only meant to be a starter module that clients can customise.

[DataBlender]

Hi @KaneG, thanks for that - it's really useful!

I've been playing around with it this morning and added a network analysis output at the end to show how the inputs and outputs are related. Once I've cleaned it up a bit I'll try and share the workflow