Alteryx Server Ideas

I would suggest a service split to simplify maintenance actions in the Alteryx Server.

Split Alteryx Service into 5 services:

 Alteryx controller

• Dependencies: Alteryx Database
• Purpose: Controls job submissions
• Meaning: The Alteryx controller Service stopped means neither the users or the scheduler can submit new jobs

Alteryx Database

• Dependencies: N/A
• Purpose: Database
• Meaning: The Alteryx Database Service stopped means all the platform is down

Alteryx Gallery

• Dependencies: Alteryx Database
• Purpose: Controls the Web UI
• Meaning: Alteryx Gallery Service stopped means the gallery web page is down

Alteryx Scheduler

• Dependencies: Alteryx controller, Alteryx Database
• Purpose: Controls the scheduled jobs
• Meaning: The Alteryx Scheduler Service stopped means the users can still submit jobs manually but scheduled jobs wont be submited

Alteryx Worker

• Dependencies: Alteryx controller, Alteryx Database (only applicable if the server where the Alteryx Worker Service is running the Alteryx controller)
• Purpose: Executes the jobs
• Meaning: The Alteryx Worker Service stopped means no job will be executed on the node

This service split will help perform some maintenance tasks and enables the Platform Administrators to shutdown the platform correctly shutting down the services in the correct order.

Usage Example 1 (Cold Backup):

• Stop the controller so no more jobs can be submitted
• Stop all the workers
• After all the workers have stopped the Alteryx Database Service and the rest of the remaining services can be shutdown
• Preform the Cold Backup
• Start the services

Usage Example 2 (Changes in the worker configuration - By doing this the users will still have access to the gallery and their jobs):

• Make changes in the worker configuration
• Restart only the worker service

Usage Example 3 (Changes in the Gallery Page or Configurations):

• Stop the Gallery Service
• Make the changes
• Start the Gallery Service

It would be very useful to have a standardized method to set up notifications to the server admins and to the job owner themselves if an alteryx canvas fails (error).

My understanding is that currently the only way to do notifications on failure for every canvas is for every canvas to individually set up events.   On a large implementation, this becomes very difficult to manage.

As large enterprises continually strengthen security around their system and data assets, we're seeing adoption of products like CyberArk's Enterprise Password Vault (https://www.cyberark.com/products/privileged-account-security-solution/enterprise-password-vault/ )

The system is essentially a central repository that secures and automatically rotates passwords for privileged accounts- things like a functional account you would use to run workflows against a certain database or set of systems.

It would be great if Alteryx could build both Server (Run As Account) and Designer (for individual database connections) integrations with a tool like that.

When restoring an Alteryx Gallery instance to a second box for test & dev it's highly likely that you don't want all your workflows scheduled from your production instance to run in your secondary instance.

However there doesn't currently seem to be a kill switch that you can implement up front to stop your scheduled workflows from running. The only way to disable scheduled workflows in your test gallery is to manually delete them all, which is annoying when you have hundreds.

It would be great to have a config flag to disable scheduled workflows before the service is started.

So - one of the biggest challenges that we have with the MongoDB used by Alteryx Server is that we continually have issues with locking (where our admins have to go in and undo locks)

Additionally - the current implementation of MongoDB connectivity does not support full Kerberos authentication which means that we're on a non-compliant install (which in a large enterprise is an uncomfortable place).

Given that a very large amount of what the server does is transactional - it would make sense to have an option to use a large-scale SQL server instead of using Mongo.   For large enterprise customers, there must be flexibility to allow the databases that they have large supported instances of (my strong preference would be MS SQL 2016).

MS SQL natively supports XML so all the canvasses can be stored in native format.   Additionally, MS SQL allows very fast query across XML, and given the clustering and reporting capabilities in MS SQL, this would dramatically increase our ability to self-manage our infra.

Given that Alteryx is looking more and more at large Enterprise customers - a move to a large-scale clustered SQL env as the back-end would be a very positive move.

NOTE: as we consider DB options for a SQL backend - please consider your large-scale enterprise customers.   For example - MS SQL or Oracle or DB2 are all much more prevalent in enterprises than databases like Postgres - so it's important to focus on the enterprise support for the DB that you choose.

+ @Deeksha @avinashbonu @revathi @BenBu

This idea covers 2 things:

- Disabling certain tools for users

- Pushing out other tools

Disabling;

- If we want to disable a particular tool for everyone or for specific users - then we need to go to every single deployed machine and adjust settings or delete some DLLs.

- What we would need is to be able to take a user / group of users - and de-select certain tools or apply limits

- In our corporate environment - every designer needs to check in with the server before fully starting up.   If it cannot connect to the server, it needs to terminate gracefully

- As the designer logs in - the first negotiated conversation is "What tools and capabilities and standard default settings should I have"

Pushing Out:

- If a useful tool is brought into our environment (like the JIRA connector here: https://gallery.alteryx.com/#!app/JIRA-Connector/58d87c2feffc2a0dd0b5ed8f or the CREW macros) - we want to be able to push these out to all the users

- Again - every designer checks in with the server first

- Then negotiates tools

- then downloads any missing tools or updates new versions of existing tools.

This kind of central server capability is essential in any enterprise deployment of several hundred seats.

When we create a shared macro on the Gallery, the desire is that we are able to:

- publish this down to users simply and seamlessly

- Allow them to use these assets in their canvasses

- Allow them to simply update to the latest version of the asset on an existing canvas.

Unfortunately - right now the only way to distribute shared macros is to create a shared folder; and the only way to manage version upgrades of a shared macro is to manually find every single usage and manually upgrade.

Use case: I need to schedule a workflow to run every 2 hours each day, with the first run at 7am and the last run at 7pm

Current solution: I need to create seven daily schedules for each hour that i need the workflow to run

Idea: Add an intraday "Start Time" and "Stop Time", or "Between" to the "Minutes / Hours" schedule type, so i can create one schedule that meets the use case requirements

Example schedule with new parameters that meets use case requirements:

Start Time = 7:00 AM

Stop Time = 7:00 PM

Repeat Every = 2 hours

Start Time could default to 12:00 AM

Stop Time could default to 11:59 PM

It would be helpful to have a central Logging tool for large & complex environments.

This would be useful to allow teams to have a central way of logging data transformation errors by error-type; severity; alteryx Canvas; etc.

Right now, there's a message tool, but this doesn't provide a way to create well structured central logging across a team; especially once deployed to a server environment.

CC: @rijuthav @jithinmony @HengHe @RajK @ydmuley @revathi @Deeksha @MPistone @Ari_Fuller @Arianna_Fuller @JoshKushner @samnelson @avinashbonu @Sunder_Sriram @Rahul_Thakur @Rahul_Singh

Currently, if you want to change from Built-in to Windows Authentication, you have to:

To switch authentication types, you'll need to start from scratch.


1. Stop the AlteryxService
2. Open a Windows Folder Explorer and go to the directory you have MongoDB loaded (System Settings->Controller->Persistence->Data Folder
3. Rename or delete that folder (a simple backup would be to rename the folder, you can move it to another directory as well)
4. Open the Alteryx Server System Settings and click next to Gallery->General.
5. Select the new Authentication process you would like to use.
6. Finish the System Settings and the new database will be created.

The Gallery and Scheduler databases are closely tied.  Workflows that are uploaded, results created, and schedules created from the Gallery are stored in the Scheduler database.  So you will need to start from scratch on both the Gallery and in the Scheduler.

This is a non-starter for any company that has any amount of time invested in establishing the use of the Gallery.   Starting over is not an option, but improving security, streamlining access protocols as a part of overall data governance is a must.  Please consider fixing the authentication protocol to make it so we don't have to start over.

Thank you,

We currently have the ability to store connections on the server which protects the credentials - however this capability does not exist for APIs for Sharepoint sites etc.

Please could you extend this to cover authentication for ALL connectors?

Many thanks

Sean

cc: @revathi

Please Enable OAuth 2.0/OpenID Support for Alteryx Server  & Connect. Currently, it supports only AD , SAML . 

Current SAML has limitations, Unable to import Security groups from LDAP/AD if SAML is enabled.  

For scheduled jobs, there should be an option to include performance profiling in output logs globally.  Basically we are looking at how we can understand which custom macros are in use, and in which workflows, and one way to do this would be to get the output logs into another tool (i.e., splunk), and then set up some queries there.  It would work, but I would have to tell N developers to go set performance profiling on, re-upload their workflows, ect.  

But why not have a switch on server / gallery that just turns this feature on globally for every scheduled workflow?  

Thank you for attending my TED talk on why performance profiling should have an option to be set globally. 

Alteryx Server does a pretty good job of tracking all the information you need in order to find out who ran a workflow at a given time, even if that workflow has been deleted from the Gallery.

However, if I need to identify what that workflow actually did on a specific date, and that version is no longer available on the Gallery, it gets really hard to track down.

There are some posts throughout the community on leveraging the MongoDB to try and reconstruct this, but in some cases the workflow is "Chunked" and becomes impossible for an end user to reconstruct.

I spoke to support about this recently and they suggested that by using the Scheduler, I could schedule the historical instance of the workflow and then "Really quickly" grab it when it was reconstructed from the temporary staging folder used by our Alteryx Server.  This has a few concerns, among them, finding a way to run this workflow so that the files can be created but it has no impact on tables, for instance if the given workflow drops a table as part of its execution.  Additionally, the only way to schedule with a different set of permissions in a 1-Worker environment is to change the default "Run Workflows As" to a different user, who also needs permission to access all files on the Alteryx Server.  This can also impact any other Scheduled or Gallery job that is executed while this recovery is underway, effectively causing downtime to do file recovery.

These restrictions are specific enough as to be impractical for most organizations using Alteryx Gallery/Server.

I think this could be solved by adding to the Scheduler an option along the lines of "Download a copy of the workflow", available to administrators. Ideally this copy could include some important metadata, like when that version of the workflow was uploaded, who uploaded it, and its ID/Version Number from the Gallery.

This would go a long way towards making it easier to Audit and respond to requests for historical information about Alteryx Workflows on the Server, and since the Alteryx Engine can already recreate these workflows, I think that a basic version of this feature could simply save out that temporary file, rather than executing it.

We spent almost a full day on Friday internally, and also about 3 or 4 hours of time for 3 Alteryx Server engineers (thank you all - MattH; ZachH; WayeW) and Ben Burkholder trying to get our gallery server to start.

Error message was "

Failed to construct Gallery daemon process, process exited code <9002>

This was all that was provided in the server log files; the gallery log files; and the Windows Event Viewer.

In the end, it turned out that the issue was very simple - the MongoDB had open locks in place that needed to be deleted; and then the service came up immediately.

Several man-days of our effort; as well as the precious time of the Alteryx Engineers who gave up their Friday afternoon to help could have all been avoided if the error-message on the server failure said "Login failure on server due to open locks - please clean out locks on db.locks collection" with a link to the solution or a lookup code that be researched on the Alteryx support site.     

Please could we go through the server service and eliminate all the exit codes and replace with human-friendly error messages - that way we could allow this kind of issue to be self-diagnosed, and allow the Alteryx Engineers to spend their time building the next generation of Alteryx Server rather than diagnosing a table lock.

cc: @avinashbonu @revathi @Deeksha @BenBu @WayneWooldridge @MattH @SteveA @ZachH 

As an enhancement to the gallery, I'd like to be able to save my application parameters in the gallery in a similar fashion to when I run an app locally.  Users can view the results of an application run, but can't save the parameters that created the run.  I'll go a step further than this and ask for another enhancement.  I'd like to be able to access the questions & responses within the execution of the app to display them during reporting.t

Use case:

For a list generation application, I need to present to the user the criteria applied during the selection process.

Input Address: 49015

Type of Vehicle: Chevy

Minimum Year: 2010

Maximum Year: 2015

I've got a working solution that is okay, but not very "Alteryx-easy".  I need to have a second update attached to the interface tool to update the user response.

Thanks for your consideration,

Mark

Alteryx has its own version control built in - however in large Enterprise environments there is a requirement to be able to use the version control platform that already exists.

In other words - when you install the server - you are asked:

Do you want to use Alteryx for version control, or your own SVN; GIT or a 4th custom option based on API integration (i.e. build this yourself based on a defined API).

This is very important to ensure that enterprise assets are maintained under the regimen of control that our Infosec & Auditors need, and the investment in making sure that Version Control contains all the right discipline; governance; etc - is all done in the enterprise Version Control platform.

Can we please extend Alteryx to have this flexibility so that we can cater to large Enterprise customers, and also anyone else who has a strict governance & change control requirement with existing infrastructure already in place?

If this is built as an independant layer and separated out from the core server slightly - it will be possibly to flexibly add other VC systems later (or provide companies with the ability to build their own).

Thank you

Sean

+ @Deeksha @BenBu @avinashbonu @revathi

Hi,

As part of enterprise alteryx server deployments, would it be possible to remove alteryx server controller tokens and use asymmetric certificates (X509) instead?

Adrian

Wondering if there are any plans to change the RC4 encryption on alteryx server to something stronger (eg: AES, Blowfish etc)

Would be curious to see if anyone else in financial services has this challenge from a security perspective.

Adrian