Alteryx Designer Discussions

Find answers, ask questions, and share expertise about Alteryx Designer.
There's still time to register for the Q3 ACE Panel - Spatial Edition presentation tomorrow! | Need more information about the event? Check out the post here

Parallel simultaneous running of workflows

Highlighted
8 - Asteroid

thanks @danilang for the threads. The download of data is the main bottleneck in the process, at least for this workflow anyway, and I don't see any parallelism capability there I'm afraid. 

 

The only thing that seems able to do it is the scheduler which is presumably running the alteryxengine.exe, and means breaking the workflow into 7 smaller workflows and putting each of them in individually.

 

Although I'm all ears if you or anyone comes up with something better.

Highlighted
Alteryx
Alteryx

@Dubya wrote:

thanks @danilang for the threads. The download of data is the main bottleneck in the process, at least for this workflow anyway, and I don't see any parallelism capability there I'm afraid. 

 

The only thing that seems able to do it is the scheduler which is presumably running the alteryxengine.exe, and means breaking the workflow into 7 smaller workflows and putting each of them in individually.

 

Although I'm all ears if you or anyone comes up with something better.


I think that's going to be the only way round it if it is the inputs that are the bottleneck.

 

Alteryx Designer within a workflow doesn't have a way of running multiple inputs at once (AKA multi-threading them). So the workaround is for you to split them into different workflows and run them at the same time yourself (or via the scheduler) and ultimately achieve multi-threading in a somewhat manual fashion by creating multiple Alteryx Engine's running at once.

Highlighted
8 - Asteroid
Thanks @JoeS

I guess I needed to hear it was the only way.

Given that were throwing this to the engine, is it possible to run multiple workflows via the command line or even a batch file run by the run command tool.

Highlighted
Alteryx
Alteryx

It is possible to run via command line with the desktop automation add on to designer, or on a server.

Highlighted

If I understood this correctly a potential solution could be to have a sequence of workflows wrapped up in a runner macro workflow or as chained apps:

1. Initial prep

2. Run command tool running a batch file that triggers several jobs for the different processing steps

 

It would be much appreciated if anyone with more experience with the  would be willing to provide an example of triggering several jobs in parallel via a batch file if the idea makes sense. 

 

Maybe a feature request, that at least in some cases would solve the parallel processing issue, be to have the option of triggering multiple apps in a chained app setup (limited by configuration options for max simultaneous jobs)? I'm not quite sure about how the chained apps are executed behind the scenes, but it might be something there.. any thoughts?

 

I guess scheduling and further dependencies would still be a problem .. hmm..

 

EDIT:

Another idea would be to just use Events - Run Command referencing instead of the chained apps setup

Highlighted
8 - Asteroid

Hi @karam @Dubya @JoeS ,

 

I have a quick question, does crew runner macros works only on alteryx server? because I am unable to run it on alteryx designer (2019.3). Just curious to know if this also works on server version only similar to Run command tool.

 

Thanks!

Highlighted
Alteryx
Alteryx

@RuchikaMangla wrote:

Hi @karam @Dubya @JoeS ,

 

I have a quick question, does crew runner macros works only on alteryx server? because I am unable to run it on alteryx designer (2019.3). Just curious to know if this also works on server version only similar to Run command tool.

 

Thanks!


Hi @RuchikaMangla 

 

It's actually the other way round. CReW runner macros are supported for designer but not Alteryx Server.

 

What's the error you are getting?

Highlighted
8 - Asteroid

Hi @JoeS 

 

thanks for your prompt reply!

 

this error I get "CReW macro error: "SupportingExes\AlteryxRunner.exe" returned an error code: -1073741819" error.

 

Highlighted
Alteryx
Alteryx

Hi @RuchikaMangla 

 

On looking into the error it looks like it might be due to some missing C++ redistributables.

 

I have found this link that I'd recommend trying initially to see if you have any success: https://answers.microsoft.com/en-us/windows/forum/all/solved-error-code-1073741819-when-running-cert... 

Highlighted
8 - Asteroid

Thanks, @JoeS 

 

I will try it after installing C++ missing redistributable. Thanks for your help, Appreciate it!!

Labels