topic Re: New node on Different VM in Alteryx Server Discussions

New node on Different VM

iwilliams — Wed, 07 Jun 2023 12:59:58 GMT

Hello Alteryx Community,

We have Alteryx server set up on a virtual machine which has some physical assets needed for workflow dependencies (python, drivers/connections, controllers, etc.). We recently set up an additional node on a fresh vm. When running workflows on the new node, we ran into errors because of the missing dependencies that are located on the first vm.

As we continue to scale, this area is where we’d appreciate some tips, advice, and best practices that you’ve used. Is it possible for the second vm to access the resources on the first vm? If not, what is the best way to maintain resources from machine to machine?

Re: New node on Different VM

fmvizcaino — Wed, 07 Jun 2023 14:06:01 GMT

Hey @iwilliams ,

All your worker nodes must have all the drivers, custom python packages and everything else installed.

A few things you can do to make this process a little easier:

Use the registry to `copy and paste` the ODBC drivers from one machine to another: https://community.progress.com/s/article/6951
Use a shared folder for the Alteryx Macros
Custom R and Python packages must be installed on all machines, also after an Alteryx version upgrade
1. Keep track of all the installed packages and develop a workflow to batch-install all packages at once

Best regards,

Fernando Vizcaino

Re: New node on Different VM

iwilliams — Thu, 08 Jun 2023 15:02:01 GMT

@fmvizcaino Thank you Fernando for this advice

Re: New node on Different VM

iwilliams — Fri, 09 Jun 2023 19:00:29 GMT

Some additional resources in case anyone viewing is in a similar situation:

https://community.alteryx.com/t5/Alteryx-Server-Discussions/Adding-a-Worker-Node-on-a-VM/td-p/513472 (similar discussion)
https://community.alteryx.com/t5/Alteryx-Server-Discussions/Extract-MongoDB-Data-from-Worker-Node/td-p/705720
https://help.alteryx.com/developer-help/virtual-environment-and-dependencies (this is moreso about python sdks, but might be useful info)
https://community.alteryx.com/t5/Alteryx-Server-Knowledge-Base/Adding-a-Worker-Node/ta-p/455827

Re: New node on Different VM

apathetichell — Fri, 09 Jun 2023 19:13:13 GMT

@fmvizcaino - have you tried writing out the .reg files that you need in workflow using output data and then writing out a .bat file to call regimport?

Re: New node on Different VM

fmvizcaino — Fri, 09 Jun 2023 19:31:07 GMT

Hey @apathetichell ,

I have no idea what you are talking about. 😁 It looks too advanced to me. Could you please elaborate?

Re: New node on Different VM

iwilliams — Fri, 09 Jun 2023 21:18:50 GMT

@apathetichell I am interested in hearing more as well

Re: New node on Different VM

apathetichell — Fri, 09 Jun 2023 21:31:09 GMT

ODBC configs are stored in registry in various locations. They are simple .reg entries - for example something like:

Windows Registry Editor Version 5.00

[HKEY_LOCAL_MACHINE\SOFTWARE\ODBC\ODBCINST.INI\SnowflakeDSIIDriver]
"Setup"="C:\\Program Files\\Snowflake ODBC Driver\\Bin\\SnowflakeDSII.dll"
"Driver"="C:\\Program Files\\Snowflake ODBC Driver\\Bin\\SnowflakeDSII.dll"
"Description"="Snowflake DSII"

may comprise the file SnowflakeDSIIDriver.reg

you can create a workflow to write text to match the keys you would expect (or use cli to download them from an S3 bucket/whatever) and then write a second .bat file and run it to set up your registry.

your second .bat file would write something like:

curl -k -L -s https://sfc-repo.snowflakecomputing.com/odbc/win64/2.25.11/snowflake64_odbc-2.25.11.msi > snowflake64_odbc-2.25.11.msi
snowflake64_odbc-2.25.11.msi
reg import "Dev-Keys.reg"
reg import "ODBC Drivers.reg"
reg import "SnowflakeDSIIDriver.reg"
reg import "odbc-data-sources.reg"

in this case Dev-Keys would contain the keys you are using for your ODBC 64 set up - and include a line like:

[HKEY_LOCAL_MACHINE\SOFTWARE\ODBC\ODBC.INI\Snowflake_Dev]

These are all plain text and regimport standardizes the ODBC configs cross environment. the above sample also includes a CURL command to download the driver directly too.