Container Parallel Processing

[kiran5189]

How to Run containers parallelly in a workflow 

[Luke_C]

Hi @kiran5189 

What's the use case you're trying to accomplish? 

[CharlieS]

The containers don't have any affect on the  processing priority. Processing priority is determined by the Alteryx engine once the workflow kicks off. The engine will evaluate the tools and processes to find the most efficient way to compute and allocate resources.

If there are processes that should be executed in a particular order, I recommend building that dependency into the workflow so that priority can be recognized by the engine. If it's "parallel", then that tells me it doesn't matter which process goes first so I would let the engine work it's magic.

[danilang]

Hi @kiran5189 

The original Alteryx Engine that processes the data moving through your workflow is serial in nature.  It only processes the data one tool at a time.  You can see this in the attached workflow.  It just generates 50M rows in two separate streams

Notice from the message tools these two streams process one after the other and each of them takes about 8 seconds to run.  In the Original engine, there is no way to get these to run in parallel.

Things are different in the new AMP engine though, which you can turn on in the Workflow Runtime settings.

Here you can see that the two Initial message tools fire at the same time, then the message tools and then the two final message stream.  The other big indication that the tools are running in parallel is the fact that the entire workflow take half as much time in the AMP engine. 

There are practical differences between the two engine modes, though.  You can't just switch a workflow from Original to AMP and expect it to run identically.  The biggest differences are around the order in which records are output from a tool.  Check out this page to see the differences. 

NOTE: As I'm writing this, there is a vulnerability in the 2021.4.2.02731 version of the AMP engine and Alteryx recommends NOT using it on either Designer or Server.  See here for details

Dan        

[Alteryx_AR]

@kiran5189 - There is logic built into Alteryx to optimize execution within the Alteryx engine.  There's nothing to tell Alteryx that one container should be run before or parrallel to other