Alteryx Server Knowledge Base

Definitive answers from Server experts.

Search Gallery Workflows for Connections, Files, Queries, Tools and More

londonhanson
Alteryx
Alteryx
Created

Search Gallery Workflows for Connections, Files, Queries, Tools and More


This workflow downloads all workflows from a Server to a local machine using the V1 API endpoint /webapi/admin/v1/workflows and the V3 API Pack, then parses through the XML to search for a specified term (such as tools used, database connections, DSNs, queries, versions, etc.).
 

Prerequisites

 
  • Alteryx Server
    • Version 2021.4+
  • Alteryx Server API Key/Secret
  • API Permissions
  • Curator Role
 

Procedure

This workflow expands on a community post, "ServerRipper - Using the Alteryx Gallery API to download workflows en masse". With this workflow, Server admins can download all workflows from the Gallery (backing up all workflows at once), and search through the XML of each workflow for any term. This is useful when an admin would like to see which workflows are using a particular DSN connection, connector, workflow version, SMTP server, query, database, and more. 

How to Configure and Run:  

Be sure to install the V3 API Pack, if you haven't already. 
Download here: https://community.alteryx.com/t5/Engine-Works/Introducing-the-Alteryx-Server-v3-API/ba-p/899228

1. Once the V3 API Pack is installed, download the attached file, SearchServerWorkflowXml_OAuth2.yxzp (or SearchServerWorkflowXml_OAuth1.yxzp). OAuth 1.0 is for pre-2021.4 Servers, and OAuth2.0 is for Server version 2021.4+.

2. Ignore the import errors on the .bat files, as they will not affect the ability to run.

3. Click into the workflow Canvas. 

4. In the Workflow- Configuration pane, click the "Workflow" tab.
image.pngimage.png

5. In the "Constants" section, edit the value of the GalleryBaseURL to your Server URL (NOTE: must not include "/gallery" at the end). 

6. Also in the "Constants" section, edit the value of the SaveTo constant to a location where all Server workflows can be saved. Ensure there is enough space in this location.

7. Configure the first tool (API Authentication Macro with #5 comment) to include your Web API address, API Key, and API Secret. This is found in the "My Profile" section of the Server UI.

8. Input a term to search for within the XML of the workflows in the Text Input Tool (replace the word "Query").
image.pngimage.png

9. Run the workflow.

10. Check the boxes of "subscriptionName" and "metaInfo_name" if any errors occur on the Data Cleansing Tool, or if there are any Windows file errors.
 

How to Know What to Search

Look at the XML of a similar workflow to one you're looking for. Is there any pattern specific to what you're searching for? Here are a few examples:
  • "Query" - Returns all workflows with database queries.
  • "<AlteryxDocument yxmdVer="2022.1">" - Returns all workflows with version 2022.1.
  • "DSN=SQL Test" - Returns all workflows using the data source named SQL Test.
  • "Plugin="SharePointInput"" - Returns all workflows that contain the SharePoint Input Tool.
  • "FileName.xlsx" - Returns all workflows that reference FileName.xlsx, either on Input or Output.
 

Common Issues

"401 Unauthorized"
  • Was the API Key and Secret input into the first tool? Is the API user an admin? Do they have curator rights?

"Error: Server API Authenticate (79): Tool #47: Parse Error at char(1): Type mismatch.  Number provided where a string is required. (Expression #1)"
  • Ensure the Web API URL is input in the first tool.

"Formula (60): Parse Error at char(62): Unknown variable "id" (Expression #1)"
  • Ensure the Web API URL in the first tool has the right base address, matching the workflow constant "GalleryBaseURL". Also remove "/gallery" from the GalleryBaseURL Workflow Constant, if present.

 "Error transferring data: [URL]: Failure when receiving data from the peer"
  • The root cause of this error is currently investigated (TCPE-539), but may be related to proxies using a PAC script. A few potential workarounds are: 
    • Throttle the records in the Download Tool.
    • Use Fiddler while decrypting HTTPS traffic (Tools > Options > HTTPS > Check "Decrypt HTTPS traffic" in Fiddler).
    • Configure the proxy to send http requests to 127.0.0.1 on port 8888.
 


Additional Resources

Attachments
Comments
JonathanAllenby
8 - Asteroid

Took at the peek at the OAuth v2 version today.

Had some workflow dependency issues with may or may not be a problem, but I'm wondering if there's a typo in the Run Command tool.

JonathanAllenby_0-1686060139837.png

I suspect at the 'Command' value is missing a \ ?

JonathanAllenby
8 - Asteroid

Right right, some user feedback for you @London !

Having some issues getting past the first tool here.

 

We're trying to run the workflow on the Alteryx Server Machine, but we're getting a 401 Unauthorized error in the download tool of the 'Server API Authenticate' macro.

 

JonathanAllenby_5-1686154501439.png

 

 

It looks like we have the macro configuration set up and the workflow constants set up and we're using the API Access Key and API Access Secret from the user's Gallery profile:

 

JonathanAllenby_1-1686154361577.png

 

JonathanAllenby_3-1686154395602.png

JonathanAllenby_4-1686154441736.png

 

We cracked open the macro to manually input the values into the Text Input, whacked a Browse tool onto the end of the Download tool, and got the full return message which doesn't give a ton of insight:

 

JonathanAllenby_6-1686154581397.png

 

Any thoughts on if we might be missing something obvious here?

 

 

JALINDS
5 - Atom

@londonhanson I have been working with this tool/process. I can download workflows using the V1 API. But I found that V1 API does not have access to all workflows while V2/V3 both do. So I adjusted to using V2 or V3. but now When downloading the workflow I only can download the JSON response of the workflow meta data that the API returns.

Have you seen this V1 limitation of not seeing all workflows and the V2/V3 only downloading the Json meta info?

londonhanson
Alteryx
Alteryx

@JonathanAllenby The dependency message is not an issue. It's just mentioning that the .bat script could not be packaged with the workflow, which is expected. This does not affect the ability to run the workflow successfully. As for the second issue, the Web API URL is shown as being set to "localhost" in the screenshot. Localhost is a reference to the host machine it's currently on, so it wouldn't work outside the host machine. We recommend changing the Web API URL to match the Gallery Base URL (with /webapi instead of /gallery).  

londonhanson
Alteryx
Alteryx

@JALINDS I suspect the reason some workflows couldn't be downloaded with V1 is due to their version, which could potentially require OAuth2 vs OAuth1. Are you receiving any sort of error message, or are the V2/V3 endpoint returning the raw JSON, rather than a .yxzp file? Which API endpoint are you using? 

TurboToad
9 - Comet

Love this workflow, @londonhanson! Been working on building something similar to complete this exercise for reviewing gallery workflows. Does this only look at the top level workflow and not go into macros or chained apps?

 

Also looking forward to the resolution on this error:  "Error transferring data: [URL]: Failure when receiving data from the peer". The throttling didn't seem to work and looking into the other workarounds.

 

Thanks!

JALINDS
5 - Atom

@londonhanson thanks for your quick reply. 

when you say version do you mean the version of Alteryx that they were made with and uploaded to (onprem) server? I have check and see both API1 and API2/3 I am using are all on OAuth2. I see no error messages. 

All the APIs have similar initial response of the workflow id, user id, name of workflow etc, there is even the information of the size of downloadData. on V1 I specify the download data blob and where to download and I get the correct yxzp but with V3 I just get the JSON blob showing the API response not workflow XML.

the endpoints that I am using are:
/webapi/admin/v1/workflows

/webapi/admin/v2/workflows

/webapi/v3/workflows

 

 

JonathanAllenby
8 - Asteroid

Another quick update @londonhanson - so, I believe they were running it from their server machine but I've got them to change their Web API Address in the tool config to their Gallery Base URL + "/webapi".

With that it's coming back as 'Error transferring data: "http://<GalleryBaseURL>/webapi/oauth2/token":  Couldn't resolve host name.'

JonathanAllenby
8 - Asteroid

Hang on, they might have made a type in the Gallery URL there.

JonathanAllenby
8 - Asteroid

Progress.

 

JonathanAllenby_1-1686838670979.png

 

csh8428
11 - Bolide

It appears I installed/configured everything correctly. The workflow runs and I can see data get through the 1st download tool. All the fields in the download tool have values, but there is nothing coming out of the JSON parse tool after the 1st download tool.

 

Any ideas?

londonhanson
Alteryx
Alteryx

@csh8428 If you click the output anchor of the first Download tool, what does the DownloadHeaders field say? If it's anything other than "HTTP/1.1 200 OK", it may not have downloaded the workflow information correctly. In this case, I would re-input the authentication information, and ensure you are a Curator in your environment. Additionally, the DownloadData field should have values in it (at first glance it looks like it contains only a bracket "[", but if you double-click the value and expand the results pane, you can see more lines of data:

 

2023-07-14_16-19-12.png

 

A couple of other areas to check:

Do you have API permissions granted to your subscription?

Does your User profile have API enabled?

Your admin should be able to check these permissions.

JonathanAllenby
8 - Asteroid

Hey @londonhanson - just a heads up that the KB version of this doesn't include the attachments:

  •  
  • SearchServerInputsOutputs.yxzp
  • SearchServerWorkflowXML_OAuth1.yxzp 
  • SearchServerWorkflowXml_OAuth2.yxzp