Alteryx - Need Shared Drive/FileShare Solution - AWS Platform

Question

Hi,

In the past, when we had one “WORKER” machine, users could read/write files from the D drive of the “WORKER” machine without any issues.

When we enabled our “CONTROLLER” as a worker machine, we told them (since your scheduled workflows will run from either the “WORKER” or “CONTROLLER”), you will need to change your file paths to a “network shared drive”.  And that works fine.

NOW, we are getting rid of our NFS (network file share) and going to “SYNCPLICITY”.  I know Alteryx doesn’t support “SYNCPLICITY”.

One of the options we thought of:

1)      Regardless if the workflow runs from the “CONTROLLER” or “WORKER” read/write access to files are accessible regardless of what machine the workflow runs from.  In essence, a shared drive in the Alteryx environment….is that possible?

2)      How do others (with multiple worker machines) handle the issue of read/write data when NOT using a nfs?

3)      What are our options, if any?

Thanks,

John

TimN · Answer

Hi Brandon,

Any thoughts or suggestions on migration from test to production when Test network shares are different from Production?  I'm thinking this forces us to change the paths in the workflows when we implement in Prod.  I was trying to use a drive letter to avoid changes in the WFs and isolate RD, QC and Prod environments.  I'm no expert by any means so any instruction or suggestion is appreciated.  Our server set up - Everything is on premise.  We have controller, UI and 2 workers all on individual servers.  We have one network drive mounted on both workers where we keep macros.  I wanted to add another share with a letter designation to send output to.

Thanks,

Tim

BrandonF · Answer

Hi John,

I'll try to add some insight here.  First, there are three main components for an instance of Alteryx Server: The Controller, Gallery, and Worker.  There are many different ways you can architect an Alteryx Server instance, largely depending on how many nodes are available and how large your deployment is expected to be.  In your case it sounds like you have two nodes, and have recently added the Worker role to the main node that hosts the Controller and Gallery.  Regardless of configuration, there are a few things to take in account when running workflows on the Gallery that access network shares:

* The first consideration is the permissions of the 'Run As' user that is being utilized.  By default, the Alteryx service runs as 'System' on Windows server.  You have the ability to set a 'Run As' user in the Alteryx Server System Settings which allows our engine to run workflows as a specified user account. This allows the workflow to access data directories that may otherwise be inaccessible by the default System account.  You will need to be sure that this 'Run As' user has the appropriate R/W permissions to the network shares that are referenced in workflows and this will need to be configured for every node that has the Worker role installed.  This will ensure that whichever node a scheduled workflow happens to run on, it will be able to access the referenced directories.
* The next consideration is the UNC path of the network share that is being access and any potential mapped network drives.  For example, the UNC path for a folder could look something like this:  \InternalFileShare\Finance\Data  However some users who access that directory may have this UNC path mapped to a network drive, such as the F:\ drive for example.  This would mean that the directory \InternalFileShare\Finance\Data location may show up as F:\Data for some Alteryx users.  This is very important to pay attention to because a workflow may reference F:\Data however if the F:\ drive isn't mapped on either of the nodes that have Workers then you will receive errors running this workflow on the Gallery.  We recommend that when referencing network locations in a workflow to specify the full UNC path as opposed to using any local mapped drive shortcuts.  This will ensure the server won't have problems finding the correct directory assuming the Run As user has the appropriate permissions.

With this understanding we can look at your other questions.

1)      Regardless if the workflow runs from the “CONTROLLER” or “WORKER” read/write access to files are accessible regardless of what machine the workflow runs from.  In essence, a shared drive in the Alteryx environment….is that possible?

As mentioned above, Alteryx can connect to any network location, including a shared drive, so long as the shared drive is mapped identically on both nodes that host Workers.  Permissions will also need to be appropriate.

2)      How do others (with multiple worker machines) handle the issue of read/write data when NOT using a nfs?

Alteryx can connect to a wide variety of data sources.  You can find a full list of everything that we support connecting to here.  Some data storage solutions, such as Microsoft OneDrive, have connectors that we have built that will actually authenticate and connect to the data source to pull files into Alteryx.  So these connectors will need to be installed on all nodes that host Workers, but it will allow Alteryx to process data that lives in this repository.  Most other common data sources that we see are actual databases.  An Alteryx Server instance can definitely connect to many different types of databases, and again you just need to be sure that the connections are correctly configured on all nodes that host Workers.

3)      What are our options, if any?

I've done some quick research on Syncplicity and it looks like there are options to host this on premise or in the cloud.  To access these files it looks like they do have a native client for Windows, but I also noticed this interesting caveat:  "File browser integration for access across OSes, platforms and devices"  Depending on how they handle this, it may be possible to browse this repository in a File Explorer window.  If that is the case, then you may be able to grab a UNC path to this directory and try to use that in Alteryx.  It would depend on how authentication is handled, but if you have a full UNC path and authentication is handled by Active Directory, then there is a chance you can give the Run As user appropriate permissions to the directory and connect via the UNC path.  I have never used Syncplicity and I do not have the ability to test with this product internally but I would recommend you give this a try.

I hope this provides some clarity for you.  Unfortunately Syncplicity is not officially supported at this time but I wouldn't say you're entirely out of the fight yet. :)   Best of luck with this!

Thank you,

Brandon