Do CREW macros need to be installed on Gallery box in order to run successfully?

[mix_pix]

Hi,

I believe the answer to this question is "Yes" but I wanted to confirm and to also understand if there are viable workarounds.

Here's the situation:

- We have a workflow that uses a Conditional Runner to call a workflow that resides in Dropbox.

- The workflow in Dropbox leverages a macro that also lives in Dropbox, since the Runner macros cannot "unzip" a packaged workflow, i.e. all assets used by the workflow being called by a Runner cannot be packaged.

- This all performs fine when we run the workflow locally.  But when we save the workflow to Gallery and try to run it, we get errors because certain assets required by the Runner macro are not accessible. 

Here is what our Runner macro configuration looks like:

Even though I've selected the Conditional Runner workflow in the macro assets, I still get this error when I save to gallery:

And when I run that workflow from Gallery, I get the following error:

So, working off of the assumption that all of these files need to be packaged into the workflow, I went back to the workflow in Designer and included these files:

I was unclear how to set up the AlteryxRunner executable file into an appropriate nested folder, which I believe is the cause of the error I get on Gallery when I try to run this modified workflow.

Is there anything I am missing?  Any help/guidance here would be appreciated. :-)

[ZacharyM · Accepted solution]

Packaging macros is not necessarily recommended, as if the Macro changes, you will have to repackage and re-publish the workflows that point to that Macro.

The better route is to have someone with access to your server machine log onto the machine and open Alteryx designer and install the macros the same way that you would on a local designer instance -> User Settings -> Macros.

I personally recommend storing all 'productionalized' macros somewhere that both users and the server can get access to (like a shared drive) and then the references are identical between users and published workflows.

Let me know if this is clear as much and I can expand on the idea.

Cheers!

Zak

[mix_pix]

Thanks Zak.  That is exactly what we are trying to pursue.  Have you encountered any issues installing the CREW macros onto the gallery server?  I know they aren't officially supported tools, but they seem pretty robust.

[ZacharyM · Accepted solution]

While not officially supported, they have worked continuously for a long time. 'Officially supported' just means that with each release we specifically check them for continuity, since they were developed by CReW (a partner of Alteryx), they are not checked in such a manner, but they havent had to change or release updates for almost 2 years.

Its also worth noting that Adam Riley, one of the CReW members involved in creating them, is a principle software engineer here at Alteryx - long story short, they are stable and I recommend them to my customers regularly.

[sanjdhar]

I am encountering a similar issue:

The workflow instantiated in Runner Macro A runs standalone in the Gallery without any issues. Same with the workflow instantiated in the Conditional Runner Macro B. (See snapshot below).

I called them up in sequence, Workflow A runs first, followed by Workflow B, via a CReW macro chained sequence. 

In the Manage Workflow assets, I chose all . When I save to Gallery, I get the "can't find the macro errors":

"_externals\2\CReW_Runner.yxmc", "_externals\2\CReW_ConditionalRunner.yxmc"

"Cannot find macro CReW_Runner.yxmcbatch" etc etc.

I then tried without checking the Crew related 3 files in the Manage Workflow assets. I then get two different error messages. 

Can you please educate me what might be happening here? 

The Crew macros themselves are installed in a shared drive location which the server accesses as well as users like me. 

Have folks been able to successfully deploy workflows with Crew macros in the gallery and run them? 

[mix_pix]

Hi Sanjaya,

We placed the CReW macros in this directory (C:\ProgramData\Alteryx\Tools\CReW) on both the gallery machine and our local machines and they work perfectly.  We don't need to package them with our workflows when we save to gallery, since Alteryx automatically checks this location for any shared resources.  And so long as the macros exist in both places (under the same path), you're good to go.

[Mahadeva]

Hi,

I don't see the 'Tools' folder under the Alteryx folder in Program data. Could you please provide more info on this.

Thanks

[ScottG]

Although the CReW macros have been used with great success, I would emphasize that the CReW Runner macros in particular should be avoided on Alteryx Server.  Beyond the fact that they are not supported, their jobs will not count toward the maximum workflows allowed to run simultaneously, they don't show up in the queue, the Server Usage Report, the Server Schedule Forecast, etc. 

Instead I recommend using Batch macros to orchestrate workflow processes.  Then the workflow itself will define what comes first, second, and so on.  

[Mahadeva]

Hi @ScottG 

Can you please provide more details on how to orchestrate jobs via Batch macros.

[ScottG]

The CReW Runner macros are often used for the sequencing of workflows.  For example, Workflow A has to run (and complete successfully) before Workflow B runs (and completes successfully) before Workflow C and so on.

There already is a way to sequence activities in Alteryx.  Think of a regular workflow.  You have an Input Data that happens first.  Then after that, say, you have Filter that happens second.  Then perhaps there is a Formula which happens third.  The workflow itself is a way of putting activities in a specific order.

If you could roll up an entire workflow into a single Tool, then you should be able to connect them together in a specific order.  Fortunately, you can roll up an entire workflow into a single Tool-- in a Macro. 

Now, there is a caveat to that.  Normally workflows, and Standard Macros, run on a record-by-record basis.  So if you converted Workflows A, B, and C to Standard Macros and connected them together, each record would go through Step A, B, and C without waiting for all records to complete in Step A first before going on to Step B, then Step C.

Enter Batch Macros.  (Or Iterative Macros for that matter.)  Batch and Iterative Macros are useful for this purpose because they are guaranteed to finish in their entirety before going to the next macro.  (Remember, Standard Macros execute on a record-by-record basis.)  Although Batch/Iterative Macros are often used for looping/repetitive activities, that isn’t required.  They can execute only once if needed, in other words. 

Workflows can be chained together by rolling up each into a macro, specifically a Batch or Iterative Macro, and then using a parent workflow that connects each macro to each other in sequence. 

Each macro doesn’t necessarily have to output any data.  Each may just output a single record with a status, say, “Success”  or “Failure”. Then the subsequent macro would either execute (on “Success”) or not (on “Failure”).   Batch Macros execute one time for every record that is sent into the Control Parameter.  So if a previous macro outputs a single record that is consumed by the subsequent macro, then that next macro in the chain will execute one and only one time.

Hopefully that makes sense.  

If you have never created any kind of Macro, then I urge you to start with the basics first:  Standard Macros.

Here are some useful links regarding the use of Standard Macros:

Getting Started with Standard Macros knowledgebase article

Macros documentation

Build Your First Macro product training

Introduction to Building Macros product training

Advanced Macro Development product training

Here are some useful links regarding the use of Batch Macros:

Getting Started with Batch Macros knowledgebase article

Batch Macro documentation

Build Your First Batch Macro product training

Here are some useful links regarding the use of Iterative Macros:

Getting Started with Iterative Macros knowledgebase article

Iterative Macro documentation

Build Your First Iterative Macro product training