Caching intermediate results

[bikramt]

I know this has been suggested before, but it would be great if calculations and transformations could be cached between workflow executions.  Perhaps the browse tools could be configured as caches.  Any spot that has a browse tool fixes the value of that node between runs provided that there are no upstream tool changes. The cache could be optional (or flushed) to allow for dynamic input data that could change between executions, even if the tool chain didn't.

[TaraM]

One of our brilliant Content Engineers, @MacRo, just created a Cache Macro - check out the blog post here: http://community.alteryx.com/t5/Engine-Works-Blog/The-Cache-Dataset-Macro/ba-p/3162

[adrianloong]

This is a really useful tool and should be built into the native product such that you can 'cache' entire containers or windows to accelerate processing

[DultonM]

@MacRo's Cache Dataset macro is awesome! I agree that something similar should be a native tool in the product. My personal ideas for this concept center around having a Cache tool that operates like any other Alteryx tool: Drag --> drop --> configure --> run. Right now the current Cache Dataset macro has 2 quirks that disrupt that Alteryx-signature-pattern: having to enable/disable tool containers and having to configure the macro in 2 steps (first to use the "Read" option, then run the app and switch to the "Write" option). A Cache tool could be designed such that it automatically is in Write mode unless it doesn't see a cached dataset (then it is in read mode; this is kind of like the Cache Data option in the Input tool). If you want to re-read the data or use Bypass mode, you say so in the tool's configuration but otherwise you wouldn't have to touch the tool twice for it to take effect. If the Cache tool is in "Write" mode, Alteryx would skip upstream tools, without them having to be disabled in a tool container. This would enable users to see all the tools and annotations upstream that explain the data coming out of the Cache tool. A workflow setting could be used to control all Cache tools on a workflow simultaneously unless the tool's configuration says to ignore the workflow setting. All this being said, I applaud the current Cache Dataset macro and all that it has done to ease workflow development.

[Atabarezz]

+1 I'm square with @DultonM on a need for native tool....

Also I would like to see the tools behind the cache macro to turn into "greyscale" once they all run and fill in the bucket...

[AlexKo]

Thanks for the insight @bikramt and everyone! We're releasing a version of caching in the upcoming 2018.3 release, which allows you to right-click on a tool in the workflow and create a cache temp file. Next time the workflow is run, it would start at that cache. It also has some UI changes that show which tools have been cached.

Stay tuned! Let us know what you think once you get a chance to play with this new feature.

[AlexKo]

Caching's live in 2018.3!

Thanks for pushing this one up on the priority list for us. We hope it will help streamline your workflow development and rapid prototyping use cases.

[Atabarezz]

Awesome work...