"Run Until Here" for large workflows

[chrisha]

In some of our larger workflows it's sometime tedious to run a workflow in order to see some data, when adding something in the beginning of the workflow. Running und stopping it as soon as the tools gets a green border is sometimes an option.

It would be convenient to have an option in the context menu to run a workflow only until a specific tool.

In effect, only this specific tool has an output visible for inspection and only the streams necessary for this tool have been run - everything else is ignored and I'm fine to not see data for the other tools.

This would speed up the development of small parts in a larger workflow much more convenient.

Regards

Christopher

PS: Yes, I can put everything else in a container and deactivate it. But a straight forward way without turning containers on and off would be preferable in my opinion. (I think KNIME as something similar.)

[TerryT]

@chrisha, have you checked out the (fairly) new caching functionality?

[chrisha]

@TerryT Yes, I have. But my idea so reversed, so to speak... I don't want a starting point with everything cached upstream, but a termination point with everything run up until a specific point.

[DavidP]

@chrisha,  how about if you just remove the connection between the tool where you want the workflow to stop and the next tool? 

[chrisha]

@DavidP Sure... I can also add a container and disable it. There already ways to achieve this, no doubt. It's just a question of ease-of-use and user experience. In larger workflows with many Filters, Dynamic Selects and Unions, it'd be handy if I can quickly get data up to the point I'm currently working on.

[ThizViz]

In the meantime, I did this sort of thing by outputting to a .yxdb mid-stream and using the Runner macros to continue the workflow.

[iCFO]

This would be massively helpful to me. Ideally it could be strategically placed in larger work flows and conditionally / manually toggled on and off. This would be a much more user friendly way to do things like allow the initial data prep portion of a workflow to be run and assessed for additional cleanup needs before moving to the main workflow.

Workarounds like splitting up workflows up and using series runners, manually toggling a bunch of containers, or nesting portions of workflows inside conditionally controlled containers within macros all have serious UI downsides.

[David_Ferland]

I agree that a 'run until here' when right clicking a node in the workflow (or something similar) would be a large improvement for the user experience. Regards