Hello, I would like to know more about recipe parallelism and whether there are any recommended best practices around this. For example if I have one input with many recipes that branch from this same input. Will all recipes execute at the same time?

andrewcoe — Wed, 25 Aug 2021 10:37:36 GMT

Is it better for the recipes to be placed one after another instead?

Re: Hello, I would like to know more about recipe parallelism and whether there are any recommended best practices around this. For example if I have one input with many recipes that branch from this same input. Will all recipes execute at the same time?

AMiller_Tri — Thu, 26 Aug 2021 13:53:02 GMT

Hi @Andrew Coe? . That's an interesting question!

Regarding whether recipes would be parallel or placed one after the other - the answer lies with what you're trying to achieve.

If each recipe is creating a different report - then, by all means, they should be separated from one another. Meaning - Parallel, all directly driven from the input file.

If they rely on one another, they should be placed one after the other.

Now, when recipes are parallel to each other - they won't necessarily execute simultaneously. This depends on your environment's configuration; more often than not, they will execute one after the other - for stability purposes and giving each recipe the resources it needs. But it will still be better practice if they're disconnected in logic from one another.

To speak in broader terms - let's look at this flow:If we run outputs [Branch 2.2]+[Branch 1]+[Branch 3], the recipe called "recipe" will be run 3 times, once for each output object that's been run. This is because Trifacta makes sure that you're working with the most up-to-date data and recipes each time you're running a job. Most of the time, it's redundant work; i.e., you'd only want this to be run once and let the continuity of the flow use its output as static reference.

To achieve this, we can do what is called a "data check-point". Meaning - write the recipe's logic into a file\table, and let that be a new input from which all other recipes continue.

That way, the recipe will only be run once instead of 3 times.

So while there's the overhead of actually writing the results of "recipe" to a destination - it's usually the lesser evil than running it multiple times.

I hope that makes sense; please feel free to ask any further questions.

Thanks,

Amit.

topic Hello, I would like to know more about recipe parallelism and whether there are any recommended best practices around this. For example if I have one input with many recipes that branch from this same input. Will all recipes execute at the same time? in Alteryx Designer Cloud Discussions

Hello, I would like to know more about recipe parallelism and whether there are any recommended best practices around this. For example if I have one input with many recipes that branch from this same input. Will all recipes execute at the same time?

Re: Hello, I would like to know more about recipe parallelism and whether there are any recommended best practices around this. For example if I have one input with many recipes that branch from this same input. Will all recipes execute at the same time?