on 06-12-2013 03:24 PM - edited on 07-27-2021 11:47 PM by APIUserOpsDM
A user might want to schedule a workflow to run each day at 7:30 AM; however, the workflow takes approximately 30 minutes to run, and the data may not always be available by 7:30. To avoid rescheduling the time-consuming workflow, it is more efficient to have a smaller workflow that checks the database to ensure the data is up to date, and then executes the main workflow when the given criteria have been met.
This can be accomplished with a Test tool from the Developer category, and a workflow event.
The following is an example of a simple logical test used to determine whether or not to kick off the main workflow. If the two input dates equal each other the test workflow will return an error.
By selecting the "Events" tab and Adding a Run Command the event can be configured to begin another process upon successful completion of this module. Configuration guidelines are:
Run Event When - Choose "After Run Without Errors"
Command - Select the AlteryxEngineCmd.exe which is usually located at "C:\Program Files\Alteryx\Engine\AlteryxEngineCmd.exe" depending on your installation
Command Arguments [Optional] - Navigate to the module you wish to schedule upon successful completion of this module. (NOTE: the filepath and module name must not have any spaces to work properly)
By scheduling the smaller test workflow at a certain time interval (e.g., every 10 minutes) for a given period of time each day of the week, the larger and more complex workflow will only execute upon the successful completion of the basic module, saving processing time and the need to re-run a complex module.
Where can I find a description of what the error codes mean?
I've been able to get mine running using this method, but always get: The external program "....AlteryxEngineCmd.exe" returned an error code: 1"
To find out the error code values you can go to the folder containing AlteryxEngineCmd.exe (mine is in "C:\Program Files\Alteryx\bin") in the command prompt and type "AlteryxEngineCmd" -- you get samples of how to use it and a description of the error codes, but here they are --
0 = Success
1 = Warnings Exist
2 = Errors Exist
There is another way that I have found and many others have found very helpful and easy.
Adam Rilely from Alteryx has built a few macros that allows you to run a module after a success of another one.
These are called Runners
You can create a workflow with that utilzes the runner to kick off the first module and then have a conditional runner to run if it's successful.
That way you don't need to worry about worry about the second module, you can then also have it run after a failure to have it rerun the module again until successful
I prefer to use this methodolgy since it puts the work back on the workflow and not behind the scenes so I can see which path the work went.
You can download these macros plus so many more on Adam Riley's sight
I hope this helps.
Levell x Dunn,
Could you be gentle enough to post an example of how to use the conditional runner macro (attached or in any way). I have tried using it with no success. It always throws out the output in the Failure section, even though I'm using the test tool inside the macro it runs and the test should yield a positive result.
Thanks!
Hi Ehatie,
As long as you're test module's work on their own then the conditional runner should be give you a positive.
Here is an example where I have 6 modules running back to back and if they error out then I have the sytem compile the error codes together via a table and then email me the results.
Can you share what error codes are you getting when it sends to the Failure section, that could help narrow down the scope and see what may be the root cause of the issue.
So more than an error code, what I keep struggling with is:
Given that to a conditional runner I input a given random set of data. This data will be taken and tested inside the workflow that the conditional runner uses. Now what I struggle to get is, why is it always sending the input data out through the failure output, if the workflow (that ends with a test tool) that is been run, works without issues and reports no failure on the test tool was given.
I have tried the conditional runner and it always fails on trying to run the 2nd item. The error seems to be because it's repeating the path twice and cannot therefor find the workflow.
Is anyone using the conditional runner in a scheduler environment? I am trying to use it and I can get the conditional runner to work if I don't UNC the module path (it doesn't have spaces). However, if I UNC the path, it looks as though the conditional runner puts the shortcut path in front of my UNC.
Ex:
Filepath: G:\my_directory\module.yxmd
UNC: \\fs30\cust$\my_directory\module.yxmd
When I put the UNC path in my conditional runner, this is the error it gives me:
Error Opening file: G:\my_directory\Alteryx Version: 10.6.8.17850¶Started running G:\my_directory\\fs30\cust$\my_directory\conditional_runner_test_module.yxmd¶Error - Alteryx Engine: Can't read the file "G:\my_directory\\fs30\cust$\conditional_runner_test_module.yxmd"¶Finished in 0.1 seconds with 1 error¶: The system cannot find the path specified. ¶ (3)
In order to schedule this, I have to UNC the paths. Anyone have any ideas?
@jm_idm Do you have the latest version of the CReW macros on the scheduler machine? I think this was a bug that was fixed.
I'm not sure...I'll check with our tech guys. Thanks!
Update:
I downloaded the version of the macros here: http://www.chaosreignswithin.com/p/macros.html
I put it in my module and it gives me the same errors.
I put the full path in the process b/c the scheduler requires it. However, it seems that the macro is adding the G:\ mapping back to the front of my path. I even copied the full UNC path and put it into the Macro so I never mapped it as 'G:\...' and still have the issue.
@jm_idm Does the Run As user have access to that network drive?
I would also suggest testing that the server can access that location.
I would try using the UNC path and attempt to open the workflow. If that works it may be the version of runner tools you have or as @SFreiter mentioned the loaded schedule job isn't being updated.
I know for us we have been successfully in using the UNC with the crew tools on the server
Can you send me a sample workflow of Runner Activity. I installed the macro and able to get the runner activity.
However, I am unable to connect one runner to other runner.
My requirement is i want to call 1.yxmd file followed by 2.yxmd then 3.yxmd.
Can you share a sample file.
Rather than use the Crew Macros, I prefer to use the built-in capability within macros, specifically Batch/Iterative Macros, for orchestrating and sequencing workflows. Although Batch/Iterative Macros are often used for looping/repetitive activities, that isn’t required. They can execute only once if needed, in other words.
Suppose you have a process to build master data which includes many read and write steps to the database.
The outputs of Step A have to be completed before Step B and that has to be complete before Step C.
I would build Batch Macros for Step A, Step B, and Step C. Each step would send out a single record, "Status=Complete" say, to be consumed by the subsequent step.
Batch and Iterative Macros are useful for this purpose because they are guaranteed to finish in their entirety before going to the next macro. (Regular macros execute on a record-by-record basis.) And Batch/Iterative Macros run in-process on Alteryx Server, whereas the Crew Macros will execute workflows out-of-process.
@ScottG
I agree that batch macros are very effective. I have over the years put together a couple of mock-ups that may be helpful to others. I cannot paste them here, but have long intended to share them with the greater Community. (I'll add a link when I do so.)
Is this type of capability available when saving workflows to Gallery/Server? Right now I am achieving this by staggered schedules for the workflows I need to run in succession, would be great if it would be possible to use this solution to eliminate unnecessary delay between workflows running on Server.