topic Re: Decrypting Passwords using Python SDK in Dev Space

Decrypting Passwords using Python SDK

chrisha — Tue, 17 Jul 2018 16:21:53 GMT

In my plugin GUI created with the HTML GUI SDK, I have a field for passwords:

// --- main.js ---
// ...

function setupItem(manager, AlteryxDataItems, name, element, password = false) {
    // Create a new DataItem for Alteryx
    var dataItem = new AlteryxDataItems.SimpleString(name)
    dataItem.setValue("")
    dataItem.setPassword(password)

    // Add DataItem to Alteryx Manager
    manager.addDataItem(dataItem)

    // Connect input to DataItem
    element.change((event) => dataItem.setValue(element.val()))

    // If we have a password, we need to asynchronous get the password
    dataItem.registerPropertyListener('value', function(propertyChangeEvent) {
        element.val(propertyChangeEvent.value)
    })
}

Alteryx.Gui.BeforeLoad = (manager, AlteryxDataItems, json) => {
    setupItem(manager, AlteryxDataItems, "userPassword", $("#password"), true)
}
Alteryx.Gui.AfterLoad = (manager) => {}

The password is stored, persistent and I can easily use it in the HTML frontend for connecting to an API.

My backend, however, is supposed to rely on the Python SDK in which I cannot get the decrypted password:

    def pi_init(self, str_xml: str):
        """
        Called when the Alteryx engine is ready to provide the tool configuration from the GUI.
        :param str_xml: The raw XML from the GUI.
        """
        setting_tree = Et.fromstring(str_xml)

        # Getting the user-entered settings from the GUI
        self.username = setting_tree.find("userEmail").text
        self.password = self.alteryx_engine.decrypt_password(setting_tree.find("userPassword").text, 0)

        self.alteryx_engine.output_message(self.n_tool_id, Sdk.EngineMessageType.error,
                                            'Password: {}'.format(self.password))

Sending the decrypted password through the output message is certainly wrong, but used here only for testing purposes. The documentation (https://help.alteryx.com/developer/current/Python/use/AlteryxEngineClass.htm) gives me three options for the decryption. Value 0 as above does not work, yielding the error

Error: Data Hub Import (v1.0) (2): Traceback (most recent call last):
File "SKOPOSDataHubImport_v1.0Engine.py", line 43, in pi_init
RuntimeError: DecryptPassword only works with User and Machine encrypted passwords.

For values 2 and 3 I get no error, but still an encrypted password.

If I read @TashaA's post here correctly, the API method for decrypting passwords should be available in 2018.2 - am I missing something?

Re: Decrypting Passwords using Python SDK

TashaA — Tue, 17 Jul 2018 22:02:04 GMT

Thanks for the heads up-we are looking into this right away.

In the meantime, could you provide the exact version of Alteryx Designer that you have?

It should be under Help->About in the menu at the top left of Designer, thanks!

@chrisha

Re: Decrypting Passwords using Python SDK

TashaA — Tue, 17 Jul 2018 23:01:04 GMT

Hey @chrisha,

It looks like the missing piece is on the HTML side. (Working with the team to get this added to the documentation.)

We need to specify the password encryption type on the HTML side as well.

Example:

var passwordDataItem = new AlteryxDataItems.SimpleString('Password', {password: true, encryptionMode: 'machine'})
manager.addDataItem(passwordDataItem)

Re: Decrypting Passwords using Python SDK

chrisha — Thu, 19 Jul 2018 14:06:31 GMT

Hi @TashaA,

thanks for following up. I have added the encryptionMode to the setup:

function setupItem(manager, AlteryxDataItems, name, element, password = false) {
    // Create a new DataItem for Alteryx
    var dataItem = new AlteryxDataItems.SimpleString(name, { password: password, encryptionMode: 'machine' })
    dataItem.setValue("")

    // Add DataItem to Alteryx Manager
    manager.addDataItem(dataItem)

    // Connect input to DataItem
    element.change((event) => dataItem.setValue(element.val()))

    // If we have a password, we need to asynchronous get the password
    dataItem.registerPropertyListener('value', function(propertyChangeEvent) {
        element.val(propertyChangeEvent.value)
    })
}

When using AlteryxEngine.decrypt_password(..., 0) this worked well. Notably, when setting the second parameter to 2 (as I would do according to the documentation), I again get an encoded string.

So, everything works now. Would be great if the documentation is updated.

Thanks a lot, Tasha!

PS: Alteryx version is 2018.2.4.47804

Re: Decrypting Passwords using Python SDK

chrisha — Fri, 27 Jul 2018 14:53:25 GMT

Quick Follow-Up question: I can use the macro just fine locally and in the Designer installed on the server. When I upload a workflow containing the tool, however, I get an error message:

Traceback (most recent call last): File "SKOPOSDataHubImport_v1.0Engine.py", line 51, in pi_init RuntimeError: Internal Error: DecryptPassword - buffer too small

Since the issue only arises when the workflow is started from the Alteryx Gallery on our internal server, I'm not sure how I can further debug this. But it seems like a problem with the SDK?

Re: Decrypting Passwords using Python SDK

TashaA — Fri, 27 Jul 2018 17:36:15 GMT

Thanks for reporting this @chrisha - logging this for our team to investigate further.

Re: Decrypting Passwords using Python SDK

chrisha — Fri, 03 Aug 2018 10:03:37 GMT

Thanks! Is there a chance we see a fix soon, e.g. with the next update for the server? Otherwise, I need to look for a workaround. :)

Re: Decrypting Passwords using Python SDK

chrisha — Mon, 06 Aug 2018 10:14:57 GMT

Another follow-up on the problem with the "Buffer too small" error: I came across this error not only on the server, but also on other machines opening a workflow containing an encrypted password for this tool. While it is shown as an error with the Python script, there already seems to be a problem on the JavaScript side: The masked password string in the HTML input is already much longer than the actual password and does not work for logging in (i.e. in the HTML GUI using JavaScript).

So, could this be an encryption key problem with the correct key only stored on the machine the workflow was originally created? If so, is this the intended behavior?

Re: Decrypting Passwords using Python SDK

TashaA — Wed, 29 Aug 2018 17:47:14 GMT

@chrisha

Thanks for following up and providing this additional information!

I reached out to the dev team and they provided this response

"There are really two-modes of encryption that we support; USER vs MACHINE. If you encrypt with USER, then it can be decrypted by USER on any MACHINE. If you encrypt with MACHINE, then only same USER same MACHINE can decrypt. For decryption errors, we will throw the same exception “Buffer too small”, so it could be a decryption error other than that and is being masked by the same exception being thrown."

For encryption information on the UI (JavaScript) side we have new documentation about the supported modes
For encryption information on the engine side (Python) we have corresponding decryption information

Re: Decrypting Passwords using Python SDK

chrisha — Fri, 31 Aug 2018 09:49:05 GMT

Hi @TashaA,

thanks for your kind and helpful response!

To my understanding that means, that it is currently not possible to use the password encryption feature in Tools when the workflow goes to the Alteryx Server. Because the user running scheduled workflows does not necessarily need to be the same user who has created the workflow. (As far as I know, our workers are running under a different AD user, because we need to access certain network drives.)

One final follow-up question: How can I use the "obfuscation" mode in the Python SDK? The documentation only lists "0: Decrypt" as option, but the initial problem in the first posts shows that this is not the same as the default mode from the JavaScript SDK. Correct?

Thanks & best regards

Christopher

Re: Decrypting Passwords using Python SDK

cam_w — Thu, 24 Jan 2019 16:09:49 GMT

Hi @TashaA

I'm hitting the exact same error message as @chrisha with my custom python tool created to connect to MongoDB with SSL (ca.pem key needed). My version of Designer is: 2018.2.6.51223. The tool is working nicely in Designer with either 'machine' or 'user' encryption.

However, my question is around the 'user' encryption on the server. I understand that I can't have someone else run my workflow on the server, but we use Windows authenication to the server and I can see that my 'Network Name' on the Alteryx Server is the same domain and user as on my Win7 laptop. However, I can't run my own workflow on the server even though it's the same 'user'. I have the password encrypted as 'user' in the GUI interface file (.html), and with decrypt_password(self.password,0) in the python engine file (.py). Both of which are in accordance with the documentation you linked.

Am I missing something? Seems like it should work if the same user is trying to run on the server. We have set the option to 'log on as batch job' on the server. Would that influence this result?

Thank you,

Cameron

Re: Decrypting Passwords using Python SDK

cam_w — Thu, 24 Jan 2019 17:38:41 GMT

Update - I have checked that the "Run Mode" of the workflow on the server was "Unrestricted", per the notice about "Python tools on Server":

https://help.alteryx.com/developer/current/Python/Overview.htm?Highlight=encryption

Also, that SDK document makes me think I should be using 'machine' encryption, not 'user'. But then I would need to pass the password to the workflow at run time on the server, which would mean using Interface controls (no scheduling?).

My server colleague has left for today, so I'll switch back to 'machine' tomorrow and try an Analytic App, but I tried with 'user' today and passed the password from an Interface text box to the custom python tool using both the masked and unmasked options. Both options returned the same error message:

Traceback (most recent call last): File "PyMongo SSLEngine.py", line 124, in pi_push_all_records RuntimeError: DecryptPassword only works with User and Machine encrypted passwords. (Tool Id: 9)

I see the following recommendation on the Python SDK page:

"Recommended for scheduling and uploading to local instances of Alteryx Analytics Gallery"

Is there any documentation on how that is supposed to work? Do we have to use Interface controls in an Analytic App? If so, how are we supposed to schedule the App with the password if it needs to be supplied at run time?

Thanks,

Cameron

Re: Decrypting Passwords using Python SDK

TashaA — Mon, 28 Jan 2019 17:57:04 GMT

Thanks for the questions! Tagging @BlytheE , she will have the most up to date information for this use case.

Re: Decrypting Passwords using Python SDK

wthompson — Mon, 28 Jan 2019 19:01:34 GMT

The encryption methodology requires that the password decryption be done on the same machine that the encryption was done. This means that the password can not be decrypted on a different machine.

If you select USER mode, only the user that encrypted the password can decrypt it and it has to be on the same machine.

If MACHINE mode is selected, then any user on the same machine where the encryption was done can decrypt the password.

The HIDE mode, is not allowed, because it does not use encryption.

Re: Decrypting Passwords using Python SDK

cam_w — Mon, 28 Jan 2019 19:31:09 GMT

Hi @wthompson,

So that would mean, if I'm understanding, deploying a tool with password encrypted to the Server would never work.

+ @BlytheE

I've also tried deploying the tool inside an Analytic App to the Server with the intent that the user encrypt and decrypt the password on the same machine (i.e. the Server). However, this resulted in the above "DecryptPassword only works with User and Machine encrypted passwords." error message.

What am I doing wrong? Is the Interface control text box with password masking causing an issue when passing that value to the custom Python tool that also does some encryption? Should I remove the encryption from the custom Python tool GUI html?

Re: Decrypting Passwords using Python SDK

wthompson — Mon, 28 Jan 2019 19:57:05 GMT

Hi @cam_w,

The only encryption mode that would work (if at all), is MACHINE mode and have the encryption and decryption done on the server side. Not sure how you would be able to do that, but that is how it would have to work.

Re: Decrypting Passwords using Python SDK

cam_w — Mon, 28 Jan 2019 21:29:16 GMT

Thanks @wthompson and @BlytheE!

It's my understanding that we can't schedule Apps where we pass the password to the custom python tool on the server, so that 'machine' encryption can be used on the server. Our only option would be to run an App version manually on the server. Or I'm going to try removing encryption from the custom python tool, saving the password in 'clear' text, and then encrypting the entire workflow when saving it to the server.

Has anyone from the Alteryx team done this? Or are we breaking new ground here?

Re: Decrypting Passwords using Python SDK

Calliecobbs — Tue, 18 Jun 2019 14:53:08 GMT

So is current state on this we still can't schedule anything to the Gallery using this functionality? I'm leveraging the ThoughtSpot connector (which leverages Python) and am getting the same error. I don't have the option of setting User vs Machine within this prebuilt tool.

Re: Decrypting Passwords using Python SDK

BlytheE — Mon, 24 Jun 2019 19:03:09 GMT

Hi @Calliecobbs that is correct. We have determined that neither encryption type can be used on the Gallery or when scheduled because they both depend on being run by the same user, which will rarely be the case in a server environment. We are looking into ways to fix this, but I don't really have a time frame for it.

Re: Decrypting Passwords using Python SDK

niravparikh — Wed, 03 Jun 2020 03:45:47 GMT

Any updates to this? BlytheE . We are running in to similar issue with PowerBI Connector