Community Spring Cleaning week is here! Join your fellow Maveryx in digging through your old posts and marking comments on them as solved. Learn more here!

Alteryx Server Knowledge Base

Definitive answers from Server experts.

Job Prioritization and Worker Node Assignment

DiganP
Alteryx Alumni (Retired)
Created

With the release of 2018.3, server users can now take advantage of job prioritization and worker node assignment. This article introduces these features and outlines how to use them.

Getting Started - User Permissions

The implementation of both of these features starts at the User permissions. Server Admins can now enable and disable scheduling, job prioritization, and worker node assignment all on a per-user basis.

picture1.png

Note: The server admin needs to set the Global Schedule setting on the Jobs page to ‘Yes’ first.

picture2.png

Server Job Prioritization

This feature allows users to assign a priority to a manually run or scheduled job to control the order jobs are picked within the queue. This means that jobs in priority order with the highest priority jobs will always execute first. If no priorities are set or jobs have the same priority level, jobs will run based on their order in the queue. Users can select from Low, Medium, High or Critical priority levels to ensure certain jobs always take priority over others.

Priorto the 2018.3 release, jobs were executed on a First In First Out (FIFO) basis. This allows admins and users to have more control over the queue execution on Server, improves efficiency and avoid bottlenecks.

picture3.png

This feature works in line with the already existing Quality of Service(QoS) feature, which is used to manage resource allocation in a multi-node deployment. For normal operation, leave this setting at 0.

  • 0 = Normal/Low
  • 1 = Medium
  • 2 = High
  • 3 = Critical
  • 4 = Chained Apps
  • 6 = Validation

Server Worker Assignment

In a multi-node server environment, the Server Admin can add job tags to workers to control which jobs are run by each worker. Users can also opt-in to allow the designated worker to run any unassigned jobs. This is done in the Worker section inside of System Settings.

picture4.png

The corresponding setting in the Gallery:

Picture5.png

If a user does not select a job tag for a workflow or job, the workflow or job will run on the default worker designated in the studio, or any worker that has been designated to run unassigned jobs.

Notes:

1) Always make sure you have one Worker Node with ‘Run Unassigned Jobs’ checked, or the un-labeled job will sit in the queue forever.

2)If a worker goes down, the jobs will be stuck in queue till its online again.

A default worker can be set and assigned at the Studio level by the Gallery Admin. This will ensure that all jobs in the studio will run on a specific worker. The user can override the default worker assigned by the Gallery Admin when scheduling or selecting a workflow to run.

Picture6.png

This setting can be found under Admin > Subscription > Username > Assigned Worker

Common Scenario Best Practices:


1) Its best practice to reserve specific workers for higher priority or critical request. Depending on the workload, this could start as small as a 2 core node.


2) Always make sure you have one Worker Node with ‘Run Unassigned Jobs’ checked, or the un-labeled job will sit in the queue forever.


3) Workers can be set in different geographical regions, NA and EU for example, if there are certain rules and regulations (GDPR etc.) that you need to abide by.

a. Customers can set a different worker with different ‘Run-As’ permissions for security or data silos purposes.


4) I’ve seen instances where different departments have their own workers - one for Finance, one for Professional services etc. This will help remove the performance bottleneck and the workload is not dependent on different department verticals.

5) For cloud environments, workers can be set on cloud environments to autoscale* resources as needed.


a. AWS Auto-Scaling, Azure Autoscale,and Google Cloud Platform Autoscale


Auto Scaling*

Auto Scaling is the process of dynamically allocating resources to match performance requirements from the infrastructure. As the demand and volume of usage grows, an application may need additional resources to maintain the desired performance levels from the nodes to meet service-level agreements (SLAs). As the usage goes down, the additional resources are no longer needed and can be lowered. This will help minimize costs while keeping the performance at scale.

The Server Dashboard (Introduced with 2018.4)

You may be asking yourself, but how can I see what are the available workers and the performance for each worker? In 2018.4, the Diagnostics page provides visibility into user and asset information on the Gallery and worker machines in an Alteryx Server environment. The server admin can use the Diagnostics page to monitor the number of users, assets, collections, studios, credentials, schedules, and jobs on the Server. You can also view the number of worker machines that are connected and identify constraints in resources impacting performance at a high level.

Picture7.png

Comments
TimothyL
Alteryx Alumni (Retired)

great article!

cliffjordan
8 - Asteroid
Thank you!
-cliff
cliffjordan
8 - Asteroid

When adding a new worker to an existing multi-node environment, does the Alteryx Service on the controller need to be restarted?

DiganP
Alteryx Alumni (Retired)

@cliffjordan You do not need to restart the Alteryx Service on the controller node. It should automatically pick it up and you should see it in the Alteryx Diagnostic Dashboard. 

afox1
6 - Meteoroid

Where can I find more information on auto-scaling worker nodes?  We are running a single server on GCP, and likely to add another.  I'd love to take advantage of scaling up and down to maximize our Cloud savings.  

DavidHa
Alteryx
Alteryx

@afox1 what cloud provider are you using?   If AWS, these two resources might be worth checking into.

 

AWS Auto Scaling - https://aws.amazon.com/autoscaling/

AWS CloudWatch - https://aws.amazon.com/cloudwatch/

 

EDIT:  I missed you are on Google Cloud.  Leaving the links above in case it helps others.

sathiya
8 - Asteroid
Thanks for the documentation Question 1 - Will the Priority , Status and other flags in Gallery Web GUI available in Alteryx Designer GUI . We schedule jobs from the Alteryx GUI and jobs are from Disk , i do not see the option in GUI for Disk or from Scheduler DB. Question 2 - AutoScaling can be done in AWS using Cloudformation files . Any advise from Alteryx to provide example Cloudformation files to create a Server , install Alteryx , Autoscale etc ..
pvd
7 - Meteor

@DiganP Thanks for the article. Can we pass in the worker tag when posting a job through API? I tried using "workerTag" in the body but doesn't seem to work. Please advice. Thanks in advance. 

DiganP
Alteryx Alumni (Retired)

@pvd Currently, API does not support worker tags - worker tag permissioning is assigned to the user, API is subscription-based. Check out this post for more information. Hope this helps.

mdemczuk
6 - Meteoroid

This is superb step forward. I would love (as curator) and user that can prfioritize workflows, to alter priorities in queued workflows. At the moment I can only do that when manually running a workflow.

ssidhpura05
6 - Meteoroid

Hello Digan,

 

Thank you for the article. You mentioned a point in notes section "If a worker goes down, the jobs will be stuck in queue till its online again" . Given a case where we have multiple worker servers and we don't want jobs to be stuck in the queue if one of the workers goes down, what will be the best practices or recommendation will have to follow to achieve this.  Any thoughts? 

lepome
Alteryx Alumni (Retired)

@ssidhpura05 
If you want high availability, avoid tagging (restricting to) specific workers.  An alternative would be to have more than one worker bear the relevant tag.

TimN
13 - Pulsar

@lepome if I have four Worker nodes (on prem installation) and I want to segregate critical work to 2 WNs and Non-Critical to the other two, the tags would be something like this?

 

WN1 Tag = Critical \ Run Unassigned = No

WN2 Tag = Critical \ Run Unassigned = No

WN3 Tag = NonCritical \ Run Unassigned = Yes

WN3 Tag = NonCritical \ Run Unassigned = Yes

 

we are contemplating this to prevent non-critical jobs from slowing our critical jobs.  Is this a good idea or can you offer a better configuration?

 

Thanks.