Re-run part of the workflow

[HM]

Hi,

I currently have a long workflow some of which link to very large datasets through an SQL server. Is it possible to re-reun only a few of the nodes instead of bringing in the same dataset every single time I run the workflow? My aim is to shorten the amount of time it takes to re-run the workflow when I'm experimenting with the nodes (it's taking around 15 minutes)

Thanks

[daniel_mmi · Accepted solution]

You might want to check out the Cache Dataset Macro, available in the Macro District, on the gallery.

http://community.alteryx.com/t5/Engine-Works-Blog/The-Cache-Dataset-Macro/ba-p/3162

It does just what it says, and lets you toggle the macro between a pass through state, in which it writes data to a temp file, and an interrupt state, in which it retrieves the same data. Once you're ready to run the whole thing again, just switch it back to get live data.

You might also want to put your slower/upstream nodes in tool containers, which you can then enable/disable.

[HM]

Thank you so much! This tool is perfect - it would be useful as a standard node in Alteryx

[ChiBK]

This seems like a great tool but I've been unable to get it to work. I start with my inputs, then have my selects, then a union. After that, I've put in the cache tool. I've run it to write a temp file, and then changed it to run from the temp file while disabling the inputs, selections, and union. I am getting an error that the cache is in read mode. 

Could anyone offer any advice?

Thanks,

Brad

[DataBlender]

Hi Brad,

The error isn't really a bona fide error:

"After you've run the workflow once with the Cache Dataset macro in "Write" mode, you can switch it into "Read" mode so that it will reference the temporary file instead of the input stream. When you do this, you'll see an error indicator over the Cache Dataset macro telling you that it's in "read mode". This is not an actual error, but is meant to be a convenient visual indicator so that when you start to build out a large workflow with many Cache Dataset macros, you can tell instantly which ones are set to "Read" mode. This is important because it means that the input datastream is being ignored, and if you make changes to an upstream tool, then those changes won't flow through the Cache Dataset macro while it is in "Read" mode since it is reading from the saved temporary dataset instead."

http://community.alteryx.com/t5/Engine-Works-Blog/The-Cache-Dataset-Macro/ba-p/3162

David

[ChiBK]

This was half the battle. I also had to write to a manually selected location, and then read from there. Once I did that, the cache worked perfectly.