Allow caching of multiple tools at the same time

[MichaelMains]

Some of the workflows I use have multiple inputs that can take a long time to initially load. The new cache function itself has been amazing, but there is one big drawback for me: I can't cache multiple tools at the same time.  Alteryx will allow me to eventually cache all of the tools I want cached, but it will take multiple times running the file.  This still saves me time in the end, but it feels a bit cumbersome to set up.

[patrick_digan]

+1 This would be great

[RachelW]

Will you tell me a little more about what you are thinking? What challenge are you facing with the way caching is currently in Designer? A screenshot indicating which tools you are hoping to cache at the same time would help me understand what you are requesting. 

[DavidP]

I also support this. Here's an example of a workflow where I'm investigation the relationship between data from 2 different database query and a data load from 250+ files. I had to run the workflow 3 times to create the 3 caching points. It would be great if I could hold in the Ctrl key and select all 3 cache points before right clicking and select Cache and Run Workflow.

[Claje]

@DavidP has captured this exactly.  We just did a training internally on 2018.3 and this was brought up.  The particular challenge is if we have a few large datasets that we need to read in, that take a few minutes each to read, you might need to "Cache and Run" multiple tims, which really limits the benefits.

EG if we have 5 datasources that need to be read in, which each take a minute to run:

Today:
Cache DS1 and run:
5 minute execution, DS1 Cached

Cache DS2 and run:
4 minute execution, DS1 and DS2 Cached

Cache DS3 and run:
3 minute execution, DS1,DS2,DS3 Cached

Cache DS4 and run:
2 minute execution, DS1,DS2,DS3,DS4 Cached

Cache DS5 and run:
1 minute execution, all 5 DS's Cached

Run workflow - runs in seconds.

This is a total of 15 minutes of execution before we get to the true value of the caching functionality.

In an ideal world, this is instead replaced by one step:

Select all 5 datasets DS1-5, choose "Cache and run"
Cache is created for DS1-5, and takes 5 minutes

This saves 10 minutes of unnecessary execution in this example.

[MichaelMains]

DavidP gave a great explanation for what I'm wanting.  A simple view of what I was thinking is attached.  In its most basic form, I'd like to be able to select multiple inputs and have them all be cached in a single run.  In more complicated situations, these could be select tools, formulas, etc.  The goal is still to avoid having to cache multiple times in order to cache large input files.Ideally there would be a cache and run option here, after selecting multiple items.

[dsibanov]

+1

[asamped1]

+1 on DavidP's comment.  This would be a great addition.

[RachelW]

Thanks for the comments and clarification. This is what I thought you meant; however, seeing your examples helped me understand it better. I can see where this isn't ideal. I'll consider this for future improvements to caching. 

Any other thoughts or feedback on caching? 

[Claje]

Some of this may be a separate idea, but I've written up a couple small ideas below:

Having a way to persist "cache to here" settings in a workflow, so that even if the cache doesn't persist when I close the flow, the locations I cache up to will, would be extremely helpful.  Right now if I am working on a complex flow for a week or two, every time I close the flow I have to remember where I was caching to (or save screenshots).  This is a pain point that I think could definitely be improved upon.

Similarly, having a "run cached" toggle in the workflow - Runtime configuration pane could be really helpful, so that if I need to run the workflow uncached I can easily toggle that off, and then toggle it right back on again.  This would be a lot like the "Disable all tools that write output" our "Disable all browse tools" toggles, although I think it would require some persistence of "cache to here" settings.

Having easier visibility into the logic/configuration of cached tools could also be really helpful.  Perhaps in addition to the message stating that changes will clear the cache, adding a prompt if changes would clear the cache asking "are you sure you want to do this?" could add an extra safety valve to make sure the cache is not accidentally cleared.

[spblackshaw]

I'm in the same boat where I'm looking at hours instead of minutes to troubleshoot workflows with multiple data inputs.

Having the new caching functionality as well as the previous ability to toggle each data input to be cached would be the best of both worlds: