Conditional running of workflows in Gallery

[francis_oy]

Dear Friends,

I was using CReW list runner/conditional runner quite a bit in my previous projects to chain workflows together. Now I am learning gallery APIs and would like to know if we could have a similar solution that we can chain the workflow in gallery together.

What I understand is when working with gallery API, if we call the workflow execute endpoint, the response will show the job is queued and we need to periodically call the job run status endpoint to get the job status.

I would like to learn if there is an event-driven approach to chain the workflows - i.e. if the 1st wf completes, the 2nd wf will be triggered automatically.

Thanks!!

Francis

[MarqueeCrew]

@francis_oy  ,

 does this article help to solve your challenge?

https://community.alteryx.com/t5/Alteryx-Server-Knowledge-Base/Chaining-Workflows-Together/ta-p/3857... 

cheers,

mark

[francis_oy]

Hi Mark,

This helps. But in our case, we have quite a few workflows contributed by different individuals. We wouldn't want everyone to configure the event for every workflow but would like to have a 'wrapper' workflow, which will call each of the workflows in gallery via API. those workflows might have dependencies - i.e. workflow A's output is the input for workflow B.

If the wrapper workflow can get the job status automatically that would be great.

We have achieved this using CReW. But it seems CReW is not recommended for Alteryx server usage.

[danilang]

Hi @francis_oy 

In response to your initial question, there is no event driven mechanism to chain the workflows.  You have to have some mechanism to poll the /v1/jobs/{jobId}/ API until the result indicates that the job has completed. 

You have 2 ways to set up the polling workflow, the first is an iterative macro that is kicked off in the workflow that starts the monitored job.  This macro calls the API until it returns completed.  You can use the CReW Wait A Second macro to add a delay between each iteration.    The drawback to this is that you now have 2 jobs running on the server until the1st job completes and the number of concurrent jobs allowed on your server is limited by your server settings. 

The second method is to schedule the monitoring job to run every X minutes.  This limits the amount of time the workflows run concurrently.  The drawback to this is that you now need an external mechanism to pass the Job IDs to the polling workflow. You can use a custom text file on the server itself or use a database table for this.

Dan

[francis_oy]

Thanks Dan. Looks like I have to create another monitor workflow as you mentioned.

So if I come back to my initial requirement - I need a mechanism to ensure the workflows run in sequential. Would you recommend we use CReW or we use API + a monitor workflow? It seems to me the latter approach is too much and also it waste a lot if time slots for waiting. 

Thanks

Francis

[danilang]

@francis_oy 

When Alteryx says that the CReW runner macros are not supported on the gallery, what they're saying is that if you use one of the runner macros in a workflow. they'll only help out in a support case if the workflow is  running in Designer.  If the workflow is running on a server then you're on your own.  You can always turn to the Community for help.  

I haven't found a case where they say that the runner macros are not "recommended" on the gallery, only that they're not supported.  I've used the runner macros on our gallery in the past with no issues

@MarqueeCrew is the expert on the CReW macros and their use on the gallery, and can correct me if needed

That being said, the runner macro is probably the way to go in your case

Dan

[francis_oy]

Thank you Dan. I have been using CReW macros on gallery as well. The only thing is my support team came to me recently and said if I use CReW, they will not be able to get all the logs in the server log table and also as CReW is executing the Alteryx Engine, they are not like other jobs submitted/scheduled via gallery - basically they are saying running workflows via CReW runners will let things out of control.

@MarqueeCrew - it would be great if you could provide your comments on pros/cons for using CReW runner macros on gallery/server.

Thanks

Francis