DCME key disaster recovery

[TheCoffeeDude]

I just underwent an exercise of recovering my controller in the event of a catastrophic failure. One of the steps is to recover the DCME keys (DCM Encryption keys) - which is documented here: https://help.alteryx.com/20221/en/server/install/server-host-recovery-guide/dcme-keys-to-backup.html...

This DCME recovery needs to be revisited. This document assumes that the previous controller is running. In a disaster recovery situation, this is not possible. What, if any, can be done to recover the DCME keys if the host has is completely irrecoverable?

For context, having an irrecoverable host has happened. Complete hard drive failure (showing my age), nuked virtual machine and its backups (no one paid attention to the notices that the data center was shutting down), and fire.

[fmvizcaino]

Hi @TheCoffeeDude ,

Totally agree with you. There should be an update to the Alteryx Server backup process including backing up the controller token and everything else that is missing there and I forgot 😅

https://knowledge.alteryx.com/index/s/article/Alteryx-Server-Backup-Recovery-Part-2-Procedures-15834...

I think you could add this as an idea. https://community.alteryx.com/t5/Alteryx-Server-Ideas/idb-p/server-ideas

A small addition to the backups is that they should be in a file server outside the server machine, preferably in a different cloud provider.

Best,

Fernando Vizcaino

[TheCoffeeDude]

Good call out to make sure the critical files and the token are backed up. I have those three files stored on several systems both on and offsite. MongoDB, I'm not too worried about - thank you MongoDB Atlas.

I'll submit the idea. I'm hoping that'll gain traction ASAP.

[jguerrero]

Thank for submitting the idea, @TheCoffeeDude!. Here is the idea for reference and for other folks to vote on: https://community.alteryx.com/t5/Alteryx-Server-Ideas/DCME-key-disaster-recovery/idi-p/1210374

[eadapa29]

Hello,

     I am facing a similar kind of issue for Alteryx rehydration process in Cloud. Wondering if there is any alternate way to extract the DCM master key as part of backup instead of running the transferdcme command from the source system ( which even doesnt exist )

Regards,

Eshwara

[TheCoffeeDude]

This is another situation why one must have the backup key without having a working server. I've discovered a way to "backup" the key, but it's not a proper backup nor is it a proper disaster recovery solution.

1. Spin up a new server and configure it to be a new controller, preferably in a different region.
2. Perform a disaster recovery on that controller, which requires you to use the transferdcme command on the old controller.
3. Never change the controller token on the new or old controller.
4. When all hell breaks loose, restore your MongoDB backup on that new controller.

This is not considered a true disaster recovery option, but you need to be creative when your options are limited.

[eadapa29]

Sorry to say this but spinning up a new server will not work because i wont be able to restore the DCM master key as i cannot backup the master key offline.

This is the exact scenario i am running into

DCM Connections fail after Host Recovery (alteryx.com)

[eadapa29]

The resolution given is fine but every month we kind of rehydrate the ec2 instances which means we have to request the DCM credentials from all the users

[TheCoffeeDude]

You're just confirming what I've been concerned about and hoping Alteryx will fix it. Their current process is not an acceptable solution, and there needs to be a way to extract this key from the gallery and allow us to store it in a location of our choosing to perform a proper disaster recovery solution. The sooner they can provide a solution, the better off we'll be when disaster strikes.

[eadapa29]

Hello,

     I have tried the transferdcmesecret command from the new controller which i restored everthing. It throws a wierd error.

PS C:\Program Files\Alteryx\bin> & "$AlteryxService" "transferdcmesecret=<Alteryx FQDN ORIGINAL HOST>:443,<Unecryped ORIGINAL HOST TOKEN>"
Invalid parameters. Could not retrieve original key.
PS C:\Program Files\Alteryx\bin>

Not sure what i am missing here...