How To Configure a Generic Vault in Data Connection Manager

[petrl]

With the 2024.1 release of the Alteryx Designer and Server, a new connection can be created in Data Connection Manager (DCM): Generic Vault.

What is it good for? You can utilize the DCM configuration to fetch secrets used in DCM credentials at runtime from almost any vault. You just need to build a custom script or executable that would handle authentication and secret retrieval from your vault, acting as a translation layer when DCM asks the vault for secrets. You need to build a custom interface that can translate the DCM requests so that your vault understands them and then do the same with the vault responses back to DCM. This includes both user authentication and secret retrieval.

Forming such a vault is possible through Designer and Server. For more information about Generic Vaults, see the DCM Generic External Vault help page.

How to connect to your vault

 

Prerequisites

First things first: to configure Generic Vaults, you must enable this option in the System Settings under DCM and select the Enable Generic Vaults checkbox.

If you’re using Server UI, you must also have the respective per-user permission (more on that under the Server Permissions help page).

You also need your vault set up so that it’s accessible via any programmatic interface (SDK, API, CLI). It’s useful to be familiar with the chosen interface, understand the authentication flow, know the request/response structure or have previously used the interface to communicate with the vault. Based on the selected interface, choose the scripting or programming language you want to use.

Build a script or executable

Before building a script, it’s good to understand how DCM retrieves secrets from the vault. When executing a workflow using a DCM connection with secrets stored in the vault, one request is sent to the vault for each secret. For example, if you’re storing both your username and password in the vault, one “fetch secret” request will be sent for the username and another for the password. Each request then contains authentication if the vault is set so.

DCM request structure

Each DCM request consists of the path to the executable accompanied by the Username and Password (optional, based on the vault configuration in DCM), the Vault Path (configured on the DCM credential), and the Value ID (again, configured on the DCM credential), and additional parameters if necessary.

The format structure can be represented as [Executable Path] [Additional Parameters] vaultPath [Vault Path] valueId [Vault Value ID] uid [Credentials.Username] pwd [Credentials.Password] or as illustrated in the image below.

The parameters can vary based on the chosen authentication method, but other than that, DCM always uses the same request structure. Your vault, however, may have various needs, format requirements, and even authentication; All of that can be and needs to be addressed inside the script/executable. You can change the order of the parameters, omit certain values, add prefixes to those values, or add your own business logic before forwarding the request to the vault itself.

Accepted response structure

DCM only accepts a textual (string) value returned from the executable. If your vault returns a JSON object (or anything apart from the secret value), you’ll need to translate the response within the script before returning it to DCM.

Examples

To give you a better idea of how to configure your vault, we’re happy to share a few examples that worked for some of our vaults, at a certain point in time. These are all supposed to serve only as examples and it is strongly advised not to use them to access your vault without properly understanding them. Be aware these are unlikely to work on your vault. Even if you’re trying to access the same vault we did, no two vault instances are identical, and the same applies to the environments used to access them.

For our use case, we used Python as the preferred language. Hence, most examples consist of a Python script and are simply calling the Python.exe executable with the script path as a parameter. The one exception is CyberArk PAM where we used Powershell to communicate through their CLI SDK.

HashiCorp Vault example

Script: hashicorp_kv1 (Python, attached below)

Required libraries:

• hvac>=2.1.0
• jsonpath-ng

Two specifics for configuring the Vault.

1. The Executable Path, when using Python, simply points to the Python executable in the installation folder, e.g. C:\Python310\python.exe.
2. We also need to give Python the path to the script. To do that, we’ll use the Additional Parameters field. It only accepts JSON array format, so even though we only have a single parameter, it has to be inside an array, e.g. ["C:\scripts\hashicorp.py"]

Now, we’re able to reference this vault when creating Credentials, just as you would with any of the previously supported external vaults. And since the script was built against a KV secrets engine, it’s able to parse the JSON response from the Vault and send only the string secret value back to DCM.

CyberArk Conjur example

Script: cyberark_conjur (Python, attached below)

Required libraries:

• conjur_api>=0.1.0
• async_timeout>=4.0.3
• jsonpath-ng

CyberArk Conjur provides its own Python library, simplifying our script quite a lot.

For our Conjur instance, we also had to provide a few additional parameters. These can certainly be included in the script directly, yet we wanted to demonstrate how to make custom parameters sent to the executable.

1. The Executable Path once again points to the Python executable, e.g. C:\Python310\python.exe.
2. The Additional Parameters field, in this case, requires not only the path to the script but also a few additional parameters, e.g. ["C:\scripts\cyberark_conjur.py", "account", "alteryx", "sslMode", "self_sign"]
   1. The example script accepts the following values as the sslMode: trust_store/ca_bundle/self_sign/insecure

CyberArk PAM Secrets Manager example

Script: cyberark_pam (PowerShell, attached below)

To obtain secrets from the CyberArk Secrets manager, we decided to use CLI SDK. This allowed us to create a simple PowerShell script, rearranging the parameters provided by the DCM to match the format required by CyberArk SDK.

As an alternative, you can use any of the SDKs provided by CyberArk, like .Net, and compile it into a single executable.

In this case, we have to provide the following:

1. The Executable path shall now point to the powershell.exe. In our case, it is C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe
2. Two additional parameters are needed: your application ID in CyberArk and the path to the script. In our case, the Additional parameters are hence configured as ["C:\scripts\cyberark_pam.ps1", "application", "alteryx-demo"]
3. Within the script, we’re interpreting the DCM credential references to vaultPath and valueId as "query" and "output field" parameters in the SDK. It is, of course, possible to have the DCM references contain any other part of the request as well.

AWS Secrets Manager example

Script: aws_secretsmanager (Python, attached below)

Required libraries:

• boto3>=1.34.41
• jsonpath-ng

For AWS, we just need to execute the Python script to make the call to AWS, with some specific authentication set up.

1. The Executable Path once again points to the Python executable, e.g. C:\Python310\python.exe.
2. The Additional Parameters field only requires the path to the script, e.g. ["C:\scripts\aws_secretsmanager.py"]
3. Due to the specific authentication to AWS, the Password field on the Generic Vault configuration has to contain a JSON object (which is then parsed in the script) in the following format: {"access key id":"<YOUR_ACCESS_KEY_ID>","region":"<YOUR_AWS_REGION>","secret access key":"<YOUR_SECRET_ACCESS_KEY>"}
   1. In our case, that meant it looks like this: {"access key id":"123123***123","region":"us-west-2","secret access key":"abcdef1234*************xYz"}
   2. This also demonstrates how to leverage the DCM credential storage in various ways to be used within the script itself.