Alteryx Server Ideas

Alteryx Designer is an amazing data tool but it’s partner, "the Scheduler" needs some much needed upgrades. The Scheduler interface that pops up from Alteryx Designer does need a complete make over. I’m not going to address this but rather focus on the functionality that if delivered makes the Scheduler much more useful. 

Today I’m reading our Mongo db scheduler data using an Alteryx workflow and Tableau to show what’s happening on the Scheduler. This dashboard is what we refer to frequently to see the health of our companies data pipeline.  I’ll share both files soon. 

Here are the top 5 features for the Scheduler.

1. Workflow priority ranking. When two or more workflows are scheduled to run at the same time, ‘priority’ sets the order of execution. Priority is set at the time of scheduling the workflow. Values range from 1 to 100. If a priority is not set the default is 50. The ‘priority’ field can be read from Mongo db. 

Why: At midnight we set off several workflows. We want to centrally manage which runs first based on a common ‘priority’ field. 

2. Restrict which controller and workers a specific workflow can run on. Default is to run on all servers. At the time of scheduling a workflow can designated to restrict which server to execute on. This creates a field called ‘restrict’ which server it cannot execute on. 

Why: Some workflows only run on the main controller due to file system references. Also a worker can be tuned for CPU or Disk I/O and workflows that can benefit from this tuning.  Selecting a disk I/O intensive workflow to run on a server tuned for Disk I/O would speed up our workflows. 

3. Set sequence of successfully completed workflows. 

FYI: We used the Runner tool for a short time to resolve this issue but learned quickly that the Runner tool is like a bull in a china shop and brought our server down. The runner tool as it is today is not an option for production work.

Why: This would allow you to run several workflows one after another. For example the first would read from a data source, the second would do calculations on the data and the third workflow would publish the data. All workflows are given a ‘workflow-number’ which can be seen in the scheduler list and read from Mongo db. 

4. If a workflow fails you can set the number of attempts to run successfully. Report attempts greater than 1 in a new field called ‘attempts’ that can be read from the Mongo db. 

Why: Some workflows fail and if attempted to run again may work. This includes issues with locked files and workflows dependent on processes outside of Alteryx. 

5. If a workflow runs more than X minutes the scheduler kills the workflow and reports a workflow error with a unique code called ‘execution-limit’.  This code can be read in the mongo db. The defaut is 90 minutes and can be set to any number of minutes. Each workflow can have it’s own limit. 

Why: Some workflows start to hog resources and need to be killed. If a new workflow is added this is a good way to protect the overall scheduled workflows. 

From a security standpoint, it is important that all users are authenticated when accesing the Gallery on Alteryx Server and using local accounts.  There shgould be an option available to force user to the login page rather than the public Gallery.  Users going to the Alteryx server URL should be presented with the login page by default before being taken to the Gallery, rather than seeing the public Gallery and needing to click Sign-in in the upper right corner.

There should be a way to set limits on Gallery so users are not able to set up to many scheduled apps to run. Currently you can either allow all users to add as many scheduled jobs to the run or none at all. If one user sets up whatever number to run, keeps other users from running their jobs. Either allow Admins to set limits or have server shift jobs around so one user cannot lock a gallery to only run his/her jobs. A combination of both a set limit per user and better job management by gallery should also solve the problem.

Hi,

We have more clients that would like to prioritize their scheduled jobs based on run time and importance. 

It could be a simple priority number, that decides which job that need to run next. Further it could be nice to be able to allocate certain workers to certain jobs. 

I am looking forward to your feedback.

Daniel

1. view by list instead of icon, can display more items per page 
2. add sorting , like sort by created date, alphabetical order
3. improve the filtering , since there is a bug. The pagination result is wrongly displayed

I am the only Curator and my name is on the 2nd page by default when no filter applied.

When the filter applied, page one showed "There are no users that fit the search criteria" , but my name is showed on the 2nd page.

I would like to request a way to manage API connections. This would mean that API connection strings can be administered in the gallery the way data connections can be created, saved and assigned in gallery. This would help in large organizations, to administer API's within specific groups. This will also help the usage report

It is more convenience that allowing gallery admin to download any workflow from the gallery for trobleshooting.

For example, if the workflow is long running , the gallery admin can download the workflow and then drill down to it to find out the root cause. Sometimes, it maybe workflow design related issue.

I have been trying to scan all of the workflows that are stored in the gallery and wrote this community post about it. Some Alteryx employees reached out to me directly to see if they could help solve my problem. We ended up with a somewhat wonky roundabout solution (that I haven't implemented yet) by downloading yxzp files through the Gallery API, unzipping them, then scanning the workflows as xml. I think the process could be greatly simplified if Alteryx had a set of Introspective tools.

The introspective tools would use similar, if not the same, processes that Alteryx already has in place to pull data from the gallery/server itself. The set of tools would be most useful for Server admins or people that are trying to build meta-workflows for Alteryx to make things easier for their users. Similar to the solution to my problem above, most of this functionality can be worked around by querying the Mongo and/or scanning engine logs. The Introspective tools would simplify this process greatly, especially when it comes to joining records from the Mongo and collecting workflow-internal data.

Since the tools will have direct access to the gallery/server it would make sense that they would only be available to machines with server licensing or could be made available if the user has a high enough permission level in the gallery they are trying to obtain data from.

Hi,

As part of enterprise alteryx server deployments, would it be possible to remove alteryx server controller tokens and use asymmetric certificates (X509) instead?

Adrian

Wondering if there are any plans to change the RC4 encryption on alteryx server to something stronger (eg: AES, Blowfish etc)

Would be curious to see if anyone else in financial services has this challenge from a security perspective.

Adrian

One of the issues that we have with Alteryx jobs in prod (and also Tableau, coincidentally) is that often a canvas is built to serve a need at the time, but after a while it's no-longer needed but it continues to run and consume server resources.

Can we add the option to our server environment to request recertification that a particular job is still needed every X months.

This will achieve 2 useful purposes:

- if the job is no-longer needed then the user hits "No thank you" and it's then taken off the scheduler which reduces server loads

- Alternatively - the user may realize that this should have been handed over to a new team or owner, and they can then make this change based on the recertification prompt.

I'm pretty certain that this would help to manage the inevitable build-up that happens on server environments where jobs build up until the server starts thrashing and the admin team then need to go out to all the users to do this recert process manually.

CC:  @rijuthav; @jithinmony; @HengHe; @RajK; @ydmuley; @revathi; @Deeksha; @MPistone; @Ari_Fuller; @Arianna_Fuller; @JoshKushner; @samN; @avinashbonu; @Sunder_Sriram; @Rahul_Thakur; @Rahul_Singh

In regulated environments, like banks, there is a requirement to fully segment Alteryx Jobs so that people on either side of a Regulatory Wall cannot access each other's canvasses or results.

As an example:

- Public vs. private side in Broker/Dealer banks

- Compliance or HR or Finance need to be segmented from all other areas in most banks

What is needed here is for any canvasses belonging to each of these walled-off areas to be controlled so that they cannot be shared across the wall, and results cannot be viewed across these areas.   This also means that the Gallery Environment needs to be capable of being segmented fully within 1 installed environment.

CC: @rijuthav @jithinmony @HengHe @RajK @ydmuley @revathi @Deeksha @MPistone @Ari_Fuller @Arianna_Fuller @JoshKushner @samnelson @avinashbonu @Sunder_Sriram @Rahul_Thakur @Rahul_Singh

It would be helpful to have a central Logging tool for large & complex environments.

This would be useful to allow teams to have a central way of logging data transformation errors by error-type; severity; alteryx Canvas; etc.

Right now, there's a message tool, but this doesn't provide a way to create well structured central logging across a team; especially once deployed to a server environment.

CC: @rijuthav @jithinmony @HengHe @RajK @ydmuley @revathi @Deeksha @MPistone @Ari_Fuller @Arianna_Fuller @JoshKushner @samnelson @avinashbonu @Sunder_Sriram @Rahul_Thakur @Rahul_Singh

In order to migrate workflows from our UAT environment to a Production server environment, we are looking for an API capability to work with the server:

- Query canvasses by name or ID (to get a list of canvasses)

- Extract Canvas to a particular location (by ID)

- Upload canvas including dependancies (with parameters for team; collection; etc)

This would assist with automating the UAT to Prod process until Alteryx Promote can step into this gap.

CC: @rijuthav @jithinmony @HengHe @RajK @ydmuley @revathi @Deeksha @MPistone @Ari_Fuller @Arianna_Fuller @JoshKushner @samnelson @avinashbonu @Sunder_Sriram @Rahul_Thakur @Rahul_Singh 

Hi,

I've got an app in my company's gallery that I've updated 11 times, so in the gallery it says "version 11". I noticed when I click on that area, it brings up a window that shows all 11 of the apps I've uploaded over time, with options to make them the published version & to download. This confused me since each time I update the app I get a warning saying the app will be deleted, which is what I want.

So for this app, I have a 800MB file within it, which would mean that on my gallery server I have 11 versions of an app that each contain at a minimum 800MB of data. I've recently been told by support that this is as it's intended, and if I want to get rid of the all the different versions, I should deleted the app entirely and start anew. This of course would change the URL to the app, requiring the updating of documentation and redistribution of that documentation to over 100 users. And I lose the number of times the app has been run, which doesn't work for me.

My idea is to have another option in the window that shows all the different versions with the options to download, but this option would be to delete previous versions...COMPLETELY, knowing that the owner would no longer have the option to download the version or make it the published version. Having this would allow the owner to maintain the original app with the option of dropping previous versions that are really not wanted anymore.

Hi,

I have described my issue in the Community post here:

https://community.alteryx.com/t5/Alteryx-Connect-Gallery/Version-Management-in-Gallery/m-p/120319#M3...

Is there a way to retrieve an exact date when new version of a workflow was uploaded to Gallery? I know it works when it is today, yesterday or couple days ago but after that you can only see "2 months ago" etc. in Creation Date, which is not sufficient for any version management.

Thank you

Alteryx Server does a pretty good job of tracking all the information you need in order to find out who ran a workflow at a given time, even if that workflow has been deleted from the Gallery.

However, if I need to identify what that workflow actually did on a specific date, and that version is no longer available on the Gallery, it gets really hard to track down.

There are some posts throughout the community on leveraging the MongoDB to try and reconstruct this, but in some cases the workflow is "Chunked" and becomes impossible for an end user to reconstruct.

I spoke to support about this recently and they suggested that by using the Scheduler, I could schedule the historical instance of the workflow and then "Really quickly" grab it when it was reconstructed from the temporary staging folder used by our Alteryx Server.  This has a few concerns, among them, finding a way to run this workflow so that the files can be created but it has no impact on tables, for instance if the given workflow drops a table as part of its execution.  Additionally, the only way to schedule with a different set of permissions in a 1-Worker environment is to change the default "Run Workflows As" to a different user, who also needs permission to access all files on the Alteryx Server.  This can also impact any other Scheduled or Gallery job that is executed while this recovery is underway, effectively causing downtime to do file recovery.

These restrictions are specific enough as to be impractical for most organizations using Alteryx Gallery/Server.

I think this could be solved by adding to the Scheduler an option along the lines of "Download a copy of the workflow", available to administrators. Ideally this copy could include some important metadata, like when that version of the workflow was uploaded, who uploaded it, and its ID/Version Number from the Gallery.

This would go a long way towards making it easier to Audit and respond to requests for historical information about Alteryx Workflows on the Server, and since the Alteryx Engine can already recreate these workflows, I think that a basic version of this feature could simply save out that temporary file, rather than executing it.

When saving a workflow to the Gallery, I would like an option on the Save window to save directly to a collection. This can be an existing collection, or would have the ability to create a new collection. The current process requires you to log into the Gallery after saving the workflow and move it to the appropriate collection.