Let cache utilize upstream cache

[patrick_digan]

I like the new cache option in 2018.3, but I would like it to function a little bit different. Let's say you cache at a certain point and then continue to build after that. If I reach another checkpoint and want to cache, it currently re-runs the entire workflow (ie it ignores my cache upstream and just goes back to the beginning of the workflow); instead, I would rather have it utilize the upstream cache. Personally, caching is usually an iterative effort during development where I keep caching along the way. The current functionality of the cache is not conducive to this. Thanks!

[Claje]

This would be a fantastic feature add for caching!

[jhueschen]

Agree completely, I feel like the 2018.3 approach to caching actually made it less functional for my purposes.  I'm actually less likely to use it vs say the crew macro or creating manual temporary outputs (which are annoying).

I would almost imagine having multiple cache "points" that each have downstream impact.  Including cache points at every data "route" would be a great way to control for time consuming data query pulls. 

[andre_geimer]

+1. That would be really helpful!

[watsacha]

Yes +1. I used SPSS Modeler in the past and I really love this feature in Modeler.

You can cache a node you want to start the workflow to run. It is very very helpful for a large workflow.

Please add this function in Alteryx. Thank you. We love you.

[AlexKo]

Thanks for the idea @patrick_digan! We're considering enhancements to caching and this one is definitely on the list.

[watsacha]

@AlexKo Thank you for the update.

Can you break Cache and Run into "Cache only" and "Cache and Run".  I can cache my input nodes before hitting run.

In addition, when I save the workflow with cache selected on some input nodes, will the cache remain when reopening the same workflow? This function works on the older versions. Thank you very much.

[RachelW]

We looked into this. Since we are creating one cache file, we don't have an upstream cache to revert to. There are some performance implications that are prohibiting us from caching on every upstream tool. 

[Claje]

Just out of curiosity, I think the thought here was that if I have cached until a certain point, and then want to cache further downstream (as more of my work becomes concrete/finalized), that the caching functionality might be expanded so that when I click "Cache and run workflow" again further downstream, it checks if that flow was already cached up to a point, and if it was, uses that cached input instead of rerunning the currently cached process, then overwrites the entire cache.

I mocked up a really simple example workflow for this:

Is functionality like this what was considered and deemed to be too complex?

Thanks!

[RachelW]

Thanks for the clarification @Claje. I didn't investigate this functionality. This might be possible. I'll keep it in the log for consideration for upcoming caching improvements. The improvement we're working on now is being able to select multiple tools and cache to multiple points. So if your workflow has multiple branches, you can cache all the branches. 

[TonyA]

Just had another customer ask about this. He has queries that can take tens of minutes to run and some very long workflows so this is causing him significant headaches.