Handling concurrency in Alteryx

Question

Hi Alteryx Community,

I'm having a concurrency problem when running multiple workflows that interact with the same database table, at the same time on the Alteryx server.

For example, when workflow_1 writes to the table dba.test and after some time, while workflow_1 is still writing, workflow_2 starts to read. Then the regular output tools write to the database in transactions, so workflow_2 will read from the state of progress, that workflow_1 is on. Workflow_1 might write in transactions of 10.000 records each time, and let's say it has written 50.000 out of 100.000 records in total. Then workflow_2 starts and reads 50.000 records which results in a false snapshot of the data.

I'm aware that this problem might relate to the database configuration itself more than Alteryx, but I'm very interested in how you have handled this issue. I would assume that some kind of locking on the table would prevent it, but I'm not aware of the best practices here. I hope that you can point me in the right direction.

Thanks in advance.

pedrodrfaria · Answer

Hi @Morten

You should set up the events to run the second WF after the first one is done running

Use the AlteryxEngineCmd executable (located in your Bin Folder) for the Command option and select the second WF file as the Command Argument. This way it will run after the first one is done running.

You should probably increase the default time out option as well.

I believe you either should use UNC Path or have that WF on your gallery machine.

Morten · Answer

How would you implement this practically. Is there an option that can make one workflow execute based on the time that another workflow completes?

The scheduled workflows consists of listruners, that holds multiple workflows.

BrandonB · Answer

Are you responsible for both workflow_1 and workflow_2? Staggering the schedules of both workflows so that the second doesn't execute until the first has completed is one approach that may work for you.