Deploying Macros In Your Alteryx Environment

[tristank]

Background

The intention of this post is to have a space where different options for deploying macros, both in Alteryx Gallery and on local machines, can be explored and compared. A lot of this documentation came from a client's request to explore how to best deploy macros to their server and Desktop users. When compiling research from a range of people and sources I decided that this could be a good reference for all Alteryx users and decided to make a quick blog post. The page will be covering a range of topics. If there is something else you would like explored or I was totally wrong on something then feel free to reach out! Any feedback is much appreciated. 

Primary Considerations

How an organization deploys and maintains macros depends on a range of factors, such as who will be using the macros, the frequency that they are updated, etc… It's important to have an understanding of an organization's primary considerations before exploring different ways to deploy macros. Listed below are some basic considerations to start with before deploying macros.

Consideration 1: Macro Update Frequency

• Macro update frequency relates to how often an organization will be updating their macros
• There are two parts to this that need to be addressed: Handling versioning (both as it relates to naming conventions and storing different versions) and making access dynamic (avoiding the use of packaged, static macros)

 Versioning

• As SDLC practices become more used within organizations how we version and track workflow / macro changes increases in relevance. We will briefly explore different options that can make that information more easily accessible to users such as through using the meta info tab or Workflow Versioning in Gallery
• When relying on packaged workflows, if a user wanted to roll back and use an older macro version they would be unable to do so (not dynamically at least). We will be exploring how we can store and manage different versions of macros so that versioning doesn't just exist as it relates to naming conventions

 Dynamic Access

• A large part of making it easy to update and quickly deploy macros is making access to the production version of the macro open and consistent for all users that may need it. This will be explored when we discuss different options for storing macros and best practices that can avoid the manual updating of workflows where macros have been changed

Consideration 2: Storage Preferences

• There is an ongoing discussion around where macros should be stored by an organization. Alteryx itself recommends using a shared drive for dynamic access, but there also exists a growing preference to shift towards storing items within the cloud, whether that be Sharepoint or One Drive, as opposed to a local or shared drive. All of these options will be explored within the page so that we will be able to compare the pros and cons of each one. Part of this exploration will include related issues such as:

  - Storing different versions

  - Which preferences allow for dynamic access

  - Git / repositories

Consideration 3: SDLC

• Any plan for deploying macros should consider the SDLC process that many organizations already follow or are working on implementing. The main considerations for SDLC are:

  - Storing macros in different repositories

  - Moving macros from one repository to another as they move through SDLC

  - Updates to macros that will require them to move through the cycle again

  - Ensuring that users are able to only deploy production stage macros

Consideration 4: Deploying Macros

• This kind of relates to storage preferences / dynamic access. It is important to consider how users are actually accessing and using macros. Initial concerns that came to mind when researching this were:

  - Static, packaged macros versus dynamic macros (or both)

  - Deploying macros to Gallery, especially when they exist within workflows

  - Deploying macros to local machines

  - Creating a macros section in the tool palette by creating a macro repository

  - Relative paths vs shared drives

Consideration 5: Future Plans

• There is an understanding that future enhancements to Alteryx such as the reintroduction of the Gallery APIs or the full development of SDLC practices, may impact how an organization wants to handle macro (or even workflow) deployment. One of the primary concerns one should have when planning out this process is how the foundational deployment can be used to ease a potential future transition to a more automated process. Specifically, an organization should design this process in a way that will make it easy to transition to using APIs and Alteryx to:

  - Using XML of macros to update versioning / track changes

  - Push macros automatically from one repository to another when ready

Industry Best Practices

I have been talking to multiple Alteryx experts within my organization and doing some research online to get a better understanding of how different businesses handle macro deployment. As a result of this research, I have gained a high-level understanding of best practices that I would like to share here before specific approaches / different options are explored in detail. Please note:

• I am not suggesting that this is how macro deployment should be done. Rather, having a high-level understanding of industry best practices will be helpful to an organization before different options are explored
• How a business deploys macros / workflows is highly dependent on a variety of factors such as:

  - How many people use Alteryx within the business

  - How the business commonly stores files

  - Whether macros need to be updated frequently after being deployed

  - SDLC practices

A quick synopsis of best practices:

• Many businesses that deploy macros do not intend on updating them / making changes to them once they exist in Gallery very frequently. As a result, the macros tend to exist within packaged workflows when moved to Gallery. Being packaged, the macros are static but when an Alteryx user pulls it to their local machine they have access to the macro and could open it and change it as needed. Macros, which themselves are workflows, can be uploaded to Gallery so that users can pull them to their local machine and use them when needed. Organizations will at times store the macros in a collection so that users can download them and implement them in their workflows.
• For businesses that don't want static macros the best practice seems to be shared drives. There are multiple reasons why storing macros in a shared drive is beneficial:
• Everyone can easily access the macros. The individual users on local machines don't have to download a package of macros or pull them from the Gallery to access them. They can just point their Designer to the shared drive and will have access to the most updated macros.
• If they have created a separate Macro section in the tool palette and pointed the macro repository to the shared drive then whenever a macro is added to the share drive it will exist within their local Designer.
• The macros can be easily updated. When uploading a workflow to Gallery by default the macro is stored within a packaged workflow, but there are alternatives to this. A user can instead identify a folder path to the macro. When this is done, every time a workflow is run in Gallery it looks to the path to execute the macro part of the workflow. If the path is a shared drive then when the macro gets updated within the shared folder the workflows that use the macro will run on Gallery using the updated macro.
• If many workflows use a given macro the shared folder is a good option because then the user doesn't have to manually update all of the workflows.
• Workflows will break if the folder where the macro was stored was changed or had a name change. (This is why versioning can't or shouldn't be part of the production macro name with exceptions)

Possible Approaches to Macro Deployment

This section will cover different potential approaches to deploying macros. Each approach will include a high-level summary and a section that details whether the approach is conducive to the primary considerations previously outlined.

Approach One: Deploying Macros Using Just Gallery

This approach would entail creating collections to store macros in the gallery. Users would be able to access the production macro collection and download the macros to their local machines. Since workflows in Gallery with macros in them cannot reference macros that exist within the Gallery, the macros would have to be packaged with the workflow when deployed to the Gallery. Once macros were downloaded to the local machine, they could be placed in a specific folder that could be referenced within Alteryx to create a tool palette of macros. Let us review how this approach aligns with our primary considerations:

Macro Update Frequency

• Not effective for frequently updating macros
• Would require that workflows get pulled down and the old macro get replaced before republishing

Storage Preferences

• Good for storage in that everything exists within the Gallery or locally on the user's machine (does not require shared drives)
• Wouldn't rely on on cloud storage in any way unless people were pulling down macros and storing them there

SDLC Practices

• Would just use the Gallery to store macros in different collections and move them through the SDLC process
• Storing different versions would have to happen in Gallery's versioning system

Deploying Macros

• Macros would have to be statically deployed to Gallery and exist within packaged workflows
• Users would have to pull down macros from the Gallery

Future Plans

• With everything existing in Gallery, it would be very possible to use the Alteryx API and other features to move macros to new collections, update macros within workflows, and store versions but it would likely be a lot of work to build it all out

Approach Two: Deploying Macros Using Gallery and a Shared Drive

Approach two would deploy macros to the various SDLC collections in the Gallery but instead of packaging the macro within the workflow, the macros would exist within a shared folder that the workflows in the Gallery could reference. The shared drive wouldn't have to have a folder for each cycle (unless we wanted a folder setup that matched the collections we have in Gallery). That said, it would be necessary that within the shared drive there exists a production folder for the macros (that the workflows can reference) and preferably a folder for storing different versions of the macros (unless the organization wanted to do versioning within Gallery itself). The main benefits of this are that any Alteryx user could access the production macros from the shared drive and the Gallery itself could reference the newest versions of the macro. Users would not have to open workflows and switch out the macros when they're updated. Users could reference the production macros folder in their Designer to get a tool palette of all the updated macros.

Macro Update Frequency

• It would be very easy to update macros within workflows, especially the ones in Gallery
• Note: Production macros would have to have a name without versioning so they could easily be overwritten and called by the workflows
• Justification for storing the versions in another folder

Storage Preferences

• This approach may not align with many organization's long-term goals of shifting to cloud storage
• Users would not have to store anything locally to use the macros

SDLC Practices

• Macros could still be deployed to collections in Gallery, copied and pasted as they move through SDLC but in the end, the production macro would have to exist in a shared drive
• Macros in a theoretical Production collection in Gallery would never be referenced in workflows (this isn't possible with Alteryx) but would still be a good reference for SDLC
• Versioning would best be done in a folder in the shared drive or using Gallery features

Deploying Macros

• Macros would be deployed, both to the gallery and to local machines, through a shared folder. They would be dynamic.

Future Plans

• Could automate the movement of the macro workflow from one collection to another during the life cycle
• Fortunately would not require modifying workflows within Gallery
• e.g. Changing the XML of the macro when it has a new version

Please note it has been difficult for me to get information on (or test) approaches three and four. If you are interested in the next two approaches please take some time to test around in a sandbox environment.

Approach Three: Deploying Macros Using Gallery and SharePoint

There are a few things to note about SharePoint before we explore using it to deploy macros. One is that there are two types of SharePoint: SharePoint Online and SharePoint On-Premises. SharePoint Online has file paths represented by URLS, and everything exists within the cloud. SharePoint On-Premises or SharePoint Server has the organization install and manage the infrastructure themselves. With On-Premises users "access files and resources by connecting to the local SharePoint server or network location using file paths specific to that environment." I am going to operate under the assumption that most organizations are using SharePoint Online. The second thing to note is that Alteryx has tools that can connect to and output to SharePoint. That said, it seems to be limited to common files for storing data/tables and is probably not relevant to deploying macros.

From what I have researched, it doesn't seem possible that macros can be stored in SharePoint and reference said macros to be used within workflows that are deployed in Gallery. The reason for this is that on the machine hosting Alteryx Server there needs to be a file path to macros that are being used in Gallery. If there is no file path, and SharePoint doesn't have file paths just URLS, then the macros need to be packaged within the workflows. SharePoint could still be used to store production macros that users could pull down to their local machines, but it does not seem like it would be an effective way to dynamically deploy macros to Gallery.

Macro Update Frequency

• Macros would need to be statically updated for each workflow

Storage Preferences

• This plan would align with an organization that desires to shift to cloud storage but at the cost of the other primary considerations

SDLC Practices

• Macros could be deployed to different collections within Gallery
• It would still be valuable to have a file for production macros and different versions or do versioning in Gallery itself

Deploying Macros

• Production macros could be easily pulled from SharePoint for users to deploy to their local Designer
• Macros in SharePoint would have to be downloaded and redeployed to workflows when updated

Future Plans

• Automating using APIs and SharePoint could be feasible
• Further exploration / testing is required

Approach Four: Deploying Macros Using Gallery and OneDrive

OneDrive differs from Sharepoint in a few ways. One is that it is primarily designed for personal storage for individual users. Two is that although it stores files in the cloud, users can create files on their local machines that can be synced to the cloud and shared with others.

Since OneDrive has actual file paths, it seems as if it is actually possible to store macros in OneDrive and dynamically deploy them to workflows in Gallery. The process would be very similar to the approach outlined in the shared drive section. There are some caveats to this approach. One is that OneDrive folders would still have to exist on the machine hosting the server and on the local machines. This is because we still need a file path for the Gallery to reference. Two is that any folder that stores production macros for deployment would have to be shared with all the users who may need to use the macros (as well as the machine hosting the server).

Macro Update Frequency

• Would be easy to dynamically update macros if we stored the production macros in a shared folder

Storage Preferences

• Workflows and macros would be securely stored in the cloud

SDLC Practices

• Could still deploy macros through SDLC using Gallery
• Versioning could be done in a folder on OneDrive or in Gallery itself

Deploying Macros

• Any user that had access to the shared folder would be able to access the most up-to-date production macros
• Gallery workflows would in theory be able to reference the macros in the shared folder

Future Plans

• Similar to the shared drive, this approach should be conducive to future automation as it relates to SDLC

Conclusion

How an organization deploys and stores macros is dependent on a range of factors. As an Alteryx Developer, it is important that you understand these factors before developing a strategy for deployment. In this post, we have explored a few ways that macros can be deployed in an organization, but it is important to note that these are not the only ways to approach this problem. Consult with members of your organization and the Alteryx community itself before executing your plan.

Extra Notes

Version Control / Git:

Version control of workflows or macros can be approached in a variety of ways. Here are some good resources to reference to gain an initial understanding of how you should be versioning:

Compare Workflows Feature

Workflow Versioning in Gallery

Managing Workflow Versions

Choosing Which Workflow to Publish

A Deep Dive into Versioning

Alteryx vs Version Control, Code Repo/Git, R and Python code, Best Practices, Promote Code

Alteryx Workflow Version Control

Version Control Best Methods

Another Alteryx Workflow Version Control Post

Resources for Macro Deployment:

Creating Your Own Tool Category in the Tool Palette

Sharing Macros in Alteryx Designer and Server

Maintaining and Sharing Macros

Best Practices for Sharing Macros