Managed promotion process for regular canvasses

[SeanAdams]

One key challenge that we have is promoting regular canvasses (not just advanced analytics models).

It would be very useful if Promote could be made part of the core product of the server:

a) split the server into 1;2;3 environments (Dev; UAT; Prod)

b) Promote then manages the workflow to promote an asset between these environments, with a configurable workflow that allows for signoffs; diff checks; control checks (e.g. are people using tools in the right way; DB connection strings etc); and then writing a copy into the firm source-code repository.

c) Promote should also then change the links in canvasses to point to a prod version of the databases and APIs if relevant.

This process is currently extremely manual and labour intensive; so if we could apply the same Promote process to regular canvasses it would be a big time saving for our admin teams.

cc: @revathi

[DavidCo]

Hi Sean, this sounds like a pretty big idea. Could you provide some clarity so the community has more context to understand the idea?

[SeanAdams]

Hey @DavidCo

(cc @Treyson @HeatherMHarris @dataMack @adrianloong)

In regulated companies such as ours - we have a SOX requirement to demonstrate appropriate control (SDLC) for any software assets (and Alteryx is included).

Key pieces of this are:

- All assets and configurations need to be under version control - preferrably in firmwide software repository

- All assets need to be tested / peer reviewed & signed off before migration (with evidence)

- Assets need to be developed off-production and should only touch production infra once tested and signed off

- Assets in production need to be traceable to the source so that they can be rolled back or reconstructed if necessary

Teams are currently filling these gaps through manual administrative effort:

- Back-copying Alteryx assets to a firmwide GIT or SVN or similar

- Workflows outside alteryx that require testing / peer review / signoff 

- Manual processes to then set these up on a prod environment

- Manual repositories to tie the prod app ID to the dev app ID.

A better approach would be:

- Alteryx server is split in to Dev / UAT / Prod

- Any asset only has one ID, which can be traded through dev / UAT & Prod

- As it is migrated (preferrably by the user) - a workflow asks the user for test evidence; and workflows it to the appropriate person (or people) for signoff

      - For us, these signoffs are:

             Dev-> UAT: Team lead signing off on peer review

             -> Prod: Line manager signing off on operational risk & testing; BI team lead signing off on prod readiness

             -> any: for any assets which have particular features - we want to send them to experts.   For example - python code needs to be peer reviewed by a Python reviewer since these skills are less prevalent.

- As it is migrated - Alteryx automatically changes the connection strings to point to prod servers

     - Many large data platforms allow you to test vs connection X in dev and point this to a dev database; and when you promote this to prod then the system flips over to the prod database.   

     - For example in dev you are developing against "devDB.Sales" with connection name "SalesConnection".    When you move this to prod, the server knows that the data connection "SalesConnecition" also has a prod version, so it flips all the assets to point to this in the prod env.

All the logs need to kept because promotion of these assets and the demonstrated evidence around this is a SOX (Sarbanes Oxley) regulatory requirement.

Note: This is not the end of the process in terms of lifecycle asset control:

• Periodic password refresh:   In our env, system passwords have to be refreshed frequently.   Alteryx Server could manage this gracefully if plugged into a credential vault
• Backout: We have to demonstrate the ability to back an asset out of prod, and return to previous state (preferably with one or two clicks)
• Leavers and movers: We are required to manage entitlement for assets - particularly in the face of new joiners; leavers; movers.   To this end - we need dual ownership of every asset at minimum; integration with entitlement control infra (two way) etc.