Split Alteryx Service into 5 Services

[joanatpsantos]

I would suggest a service split to simplify maintenance actions in the Alteryx Server.

Split Alteryx Service into 5 services:

 Alteryx controller

• Dependencies: Alteryx Database
• Purpose: Controls job submissions
• Meaning: The Alteryx controller Service stopped means neither the users or the scheduler can submit new jobs

Alteryx Database

• Dependencies: N/A
• Purpose: Database
• Meaning: The Alteryx Database Service stopped means all the platform is down

Alteryx Gallery

• Dependencies: Alteryx Database
• Purpose: Controls the Web UI
• Meaning: Alteryx Gallery Service stopped means the gallery web page is down

Alteryx Scheduler

• Dependencies: Alteryx controller, Alteryx Database
• Purpose: Controls the scheduled jobs
• Meaning: The Alteryx Scheduler Service stopped means the users can still submit jobs manually but scheduled jobs wont be submited

Alteryx Worker

• Dependencies: Alteryx controller, Alteryx Database (only applicable if the server where the Alteryx Worker Service is running the Alteryx controller)
• Purpose: Executes the jobs
• Meaning: The Alteryx Worker Service stopped means no job will be executed on the node

This service split will help perform some maintenance tasks and enables the Platform Administrators to shutdown the platform correctly shutting down the services in the correct order.

Usage Example 1 (Cold Backup):

• Stop the controller so no more jobs can be submitted
• Stop all the workers
• After all the workers have stopped the Alteryx Database Service and the rest of the remaining services can be shutdown
• Preform the Cold Backup
• Start the services

Usage Example 2 (Changes in the worker configuration - By doing this the users will still have access to the gallery and their jobs):

• Make changes in the worker configuration
• Restart only the worker service

Usage Example 3 (Changes in the Gallery Page or Configurations):

• Stop the Gallery Service
• Make the changes
• Start the Gallery Service

[CristonS]

Hi @joanatpsantos thank you for your comprehensive feedback! 

This visibility into how users would like the Alteryx Services to behave helps our product team with future planning and understanding users' needs more directly.  

[jforte]

In general, I would say that platform availability for the corporate environments (Server) needs some attention, as a high level statement.

In the end, that's what the idea is about, improve server availability.

Important to keep in mind that Server deployments, specially with hundreds of users, have very short and tight maintenance windows.

Using a DataBase engine that allows for hot backups (Trivial these days), would also make sense.

@CristonS is there any time frame to have in mind, when the product team reviews these ideas ?

PS: Me and @joanatpsantos are from the same team.

[revathi]

@joanatpsantos I support this idea .. Firms with single server setup with all four components (Controller, Gallery, Embedded MongoDB and Worker) will greatly benefit from splitting the services. 

However, all usage examples mentioned in the post above can be achieved today. 

To address usage example 1 - Hot backups are possible by setting up Alteryx underlying database in a separate server. (referring to user managed mongoDB setup). example: setup a replica set (preferably 3) in 3 Linux servers. You can then take the backup while the Alteryx servers (controller, worker and gallery installed in Windows Servers) are up and running. No downtime needed! Refer to MongoDB Backup and Restore documentation.

To address usage example 2 and 3 -  simplest solution that can be applied is to setup Controller, Gallery and Worker  - in three different windows servers. That way worker configuration can be updated without affecting the controller and gallery usage. And Gallery settings can be updated and service restarted without affecting the scheduled jobs and their execution in the workers. 

[jforte]

Dear @revathi , almost everything is possible in IT, it's just a matter of creativity, but when it comes to feasibility and TCO, now that's another story.

The suggestions given "may" be possible, but not without a lot of trouble (MongoDB is not a transactional DB, anyone who has managed MongoDB replica setup knows the amount of problems this is prone to) and expense (Alteryx Server licensing) along the way, not to mention a load more servers ?!

These suggestions are theoretically possible, but not realistically practical or feasible.

Think about it.

You suggest passing from a 1 server setup, to a 6 server setup to achieve something that is given already in most common products in the market.

Not to mention the increase on the licensing and TCO costs... ?! Not a realistic suggestion, imho.

[JohnPelletier]

Thanks, all! I hear two separate requirements here that are related.

One is to improve on our availability story. We definitely have that in our roadmap. Today we have redundancy for Gallery and Workers (and Mongo itself) and we have it in our backlog to add redundancy to the controller service.

The other is around being able to do backups without bringing everything down, or at least to have a cleaner way to bring everything down so that jobs are cleanly handled on restart. We also have this in our backlog. 

Your input is very valued, and we take it seriously.  

[TheCoffeeDude]

I moved the internal MongoDB database to MongoDB Atlas. As a result, I now have a more robust disaster recovery solution, several weeks of rolling snapshots, multi-region availability, and can easily restore those snapshots into different databases (dev, cert, or some some other individual instance). Would I recommend going to MongoDB Atlas? Absolutely, but I also know this solution isn't available for everyone.

Just my two cents about the database situation.