Engine Works
Under the hood of Alteryx: tips, tricks and how-tos.
Editor's note: Caching is now built into Designer - just right click on a tool. Check out the announcement blog.
You can download the Cache Dataset V2 macro from the Alteryx Analytics Gallery (in the Macro District) here.
Why the update?
Sometime last year, I made a tool that allowed me to create "save points" in my workflows, and avoid wasting time running the entire workflow after making incremental changes. A lot of people responded to a blog post I wrote about it, and some great suggestions came out of that discussion.
One user even made his own updated version of the macro, because he wanted to be able to specify where files were saved by the macro. That was really awesome, and inspired me to implement some of the other suggestions users had and share the updated Cache Dataset macro with you guys here.
Thanks to @ErikB for sharing his update with me! Hopefully this becomes a trend -- if you feel like some functionality is missing, don't be afraid to start poking around the guts of a macro and making your own changes and sharing them. This is good for everyone because it means we can make better tools together through collaboration, but it’s also a great opportunity to learn new patterns and tricks in Alteryx.
What's different in this version?
This update is really all about managing the storage and cleanup of cached files. In the previous version of the tool, all cached files would be saved to wherever the macro itself lived. But this wasn't always desirable, and if you use it a lot, you might end up with a big folder of files like this:
So let's take a look at what's new in the tool's configuration to deal with this problem:
For the most part, the first tab should look pretty familiar (and if not, go back to the original post to see how it works), but there are three new options under "write mode" that allow you to specify how long you want the cached files to live on. If you select the first option ("Delete after shutting down Alteryx") then the cached files will be written to (and read from) the Alteryx temp directory, which gets automatically cleared out upon closing Alteryx.
However, if you select the second option, then the files will be deleted after a specified number of days. This is triggered by running the macro, so if you use it once and then never again, then that one cached file will not be removed. But if you use the tool regularly, then it will clear out the old files.
(To accomplish this, the macro maintains an index of locations where cached files have been created, and then each time it is used in a workflow, it looks for cached files in those directories that have an expiration date that is passed due and deletes them. In a future version, I may look into automating this with schtasks.)
Finally, you have the option to never delete a cached file. (This is how the tool used to work.)
The Advanced Options tab
Here, you can tell the macro where to save the cached files. The first option ("default") will tell the macro to save to a dedicated cached_datasets folder in the user-specific APPDATA directory. However, if the "delete after shutting down Alteryx" option is selected on the previous tab, the file will be written to (and read from) the temp directory instead. The second option ("workflow directory") tells the macro to save the files to the same directory as the workflow in which you've inserted the Cache Dataset V2 macro.
Finally, if you don't want the macro to check for expired cached files and delete them, check the last option on this tab.
That's it. Enjoy!
And keep the feedback coming (and of course, your own variations)!
Teaser
I'm excited to say that caching functionality is going to be officially integrated into the product in a future release in a very elegant and streamlined way! (Some of you may have picked up on this in the product suggestions thread here.) I've had the pleasure of being involved in some of the initial design discussions around the new functionality, and I genuinely feel that it is going to be totally game-changing to how we develop workflows in Alteryx. (That's all I can share at this point, but will do my best to provide updates as they become available!) 
