Community Spring Cleaning week is here! Join your fellow Maveryx in digging through your old posts and marking comments on them as solved. Learn more here!

Alteryx Designer Desktop Knowledge Base

Definitive answers from Designer Desktop experts.

Guide to Creating Your Own Connector - Generating the Request

TashaA
Alteryx Alumni (Retired)
Created

This guideis part of the "Guide to Creating Your Own Connector" series.

Tab 1 - Basic

URL : The URL for the resource you are trying to access must come from an upstream tool and is the only field required by Alteryx to configure the Download tool. Based on the API you are trying to pull information from (or send data to), other information will be required such as headers and/or a payload.

Output: Configure the output in the way that you wish to receive it to parse downstream. If building a connector, you will most likely keep the default settings,and output to a field as a string. More information about the output and other parameters of the Download tool can be seen on the Alteryx help page.

Tab 2 - Headers

The header of a request is where you place information that must accompany all of your requests. These headers can include things like an authentication token, which indicates your authorization to make the request, as well as the type of the content that you are sending with the request. The documentation for the API you are working with should contain information on the exact headers that you are required to use in order to utilize the service.

Commonly seen headers include:

  1. content-type : application/json
  2. Authorization: Basic (encoded username : password)
  3. Authorization: Bearer (token)

Adding a header in the Download tool can be done 2 ways, and you can use a combination of both methods in the same tool.

1. From a constant value: If you know that you will be using a consistent header that will only be utilized in the current Download tool you are working in, the constant value is the way to go. This step requires that you add the key value pair directly to the Headers section. In the example below the tool will send the Header “content-type” with the value “application/json”.

ice_screenshot_20160523-111000.png

2.From a field: If you need to process your header prior to sending it, such as base 64 encoding it, or if the header relies on previous calls to the API such as a call to an authorization endpoint, use this option. Make sure that the name of the field is the name of the header you wish to send, which is the name the Download tool will utilize to send the header. In the example below, the tool will use “Authorization” as the header name, and the value of the field as the value to send.

image (2).png

Tab 3 - Payload

HTTP Action: The API that you are working with should dictate which HTTP action to use for certain endpoints. These will most often be GET to retrieve data, and POST to send or upload data, but always refer to the documentation to be sure.

Query String/Body: There are many different ways of implementing a payload/query/body with the Download tool. The payload/query is the data that is normally listed after the question mark in a URL example. Below are examples of different ways to implement this example request from the Quandl API site.

ice_screenshot_20160523-115136.png

1.Appended directly to the URL: You can take this string directly as it is and use it as the value of the URL in the Download tool. This is fine for a one time use, but in building a reusable connector, you would likely want to build the query using one of the other options so that some of the data could be parameterized. This option would require no selection on the Payload tab of the Download tool other than the HTTP Action.

ice_screenshot_20160523-115101.png

2.Compose Query String/Body:

From constant values : In this option you write out the key value pairs for the parameter and value.

ice_screenshot_20160523-115927.png

From field values: This option takes the name of the field as the name, and the value of the field as the value for the parameter that you are sending.

ice_screenshot_20160523-120623.png

You can also use a combination of both constant and from field values:

ice_screenshot_20160523-120514.png

3.Take Query String/Body from Field: Add the entire body of the query to a field and use the value of that field here. This can be a good option if your entire payload of the request has to be formatted as a JSON object.

ice_screenshot_20160523-121320.png

ice_screenshot_20160523-121351.png

4.Use Following for Query String/Body: You can directly add the query string to the Download tool:

ice_screenshot_20160523-121022.png

Tab 4 - Connection

Username & Password: These fields are most commonly used if utilizing the Download tool for FTP/SFTP.

Maximum Connections: This field defaults to 2 and has a maximum value of 32. Exercise caution when increasing this number, as to not overload the server you are accessing or exceed rate limitations of the web service.

Timeout: If you are having issues with the request timing out with the service you are accessing (this will be seen by an error message that indicates a timeout), increase this number so that you allow the service more time to respond to your request.

Additional Considerations:

Multiple Download Tool Requests: When working with the Download tool, a request will be executed for every row of data that is passed in. This makes it is a very useful practice to have all of your Download tool configurations contained in fields, so that you may use the headers and query string/body options that take the data from fields. By parameterizing this data, you can add one Download tool to your canvas and execute multiple requests. If you are attempting to create a reusable connector however, you may instead create a supporting sub-macro that executes a request and parses the data, so that the response data from multiple requests can be joined together. You should evaluate what works best for your situation based on what the desired outcome is. The example below contains the request data to retrieve 2 different data sets from the Quandl API. If fed into a Download tool, it will execute one request for each row of information.

ice_screenshot_20160524-103103.png

Rate Limitations: The service that you are working with may have limitations on the number of total requests you can execute per day, per second, per minute, etc. If you are executing multiple requests through one Download tool you will want to pay attention to these limitations. The Throttle tool is a very useful mechanism for handling these rate limitations. For example, if the API you are working with limits you to 1 request per second, you could add the Throttle tool prior to the Download tool, and use its default value of 60 to achieve the rate limitation. If you were limited to 2 per second, you could change the Throttle tool's value to 120.

Paging for Results: If you are working with a service that had the possibility of returning a relatively large set of data, the service may have a process to indicate that there are more records than what is currently being returned. This usually takes the form of something like a next page token, or a flag that indicates that there are still more records, and you may need to increment your current “page” or parse out the next page token to send back for the rest of the data. This type of behavior can be achieved by using an iterative macro. The example below shows a sample set of tools that would be contained in an iterative macro that your connector could utilize.

ice_screenshot_20160524-112652.png

The top row shows the request being made, and the JSON being parsed and output.

The bottom row shows that if a certain condition is met, such as if a next page token is present, you can do the processing that is needed, generate the new URL to include the next page token, and the macro will execute again based on this new request URL.

To configure the iterative macro settings, you use the Interface Designer window. This allows you to specify the Iteration Input tool, the Iteration Output tool, and the method to union the results. To read more about iterative macros check out the help page.

ice_screenshot_20160524-112822.png

Comments
rej
7 - Meteor

Hi @TashaA

 

The link to the pagination iterative macro example appears to be dead. 

 

Capture.PNG

 

Is it possible to get an updated link?

TashaA
Alteryx Alumni (Retired)

Thank you so much for pointing this out @rej! We will get this updated with the appropriate links. 

Hakimipous
10 - Fireball

@TashaA

 

Thanks for the great article!

 

Any news when the links will be updated? 

TashaA
Alteryx Alumni (Retired)

@rej @Hakimipous the links should all direct to the correct location now. Thank you for your patience!

Hakimipous
10 - Fireball

@TashaA 

Awesome! Thanks again!

AshishD
7 - Meteor

Hello @TashaA 

 

Thanks for the guide here. It's been really helpful for me as a newbie.

 

But I would request if you can help with my issue posted in the discussion forum:

https://community.alteryx.com/t5/Alteryx-Designer-Discussions/API-stuck-at-Download-tool-setup/m-p/1...

 

Regards,

Ashish

FrancoisP
7 - Meteor

@TashaA Thanks for your post.

 

I have a very weird behavior of the Download tool. I set the “content-type” with the value “application/json” in the Headers and add {"format":"csv"} as payload (manually type). The error I get from the server is that "HTTP message must not contain more than one Content-Type header". It looks like that Alteryx Download tool is embedding automatically “content-type = application/x-www-form-urlencoded" in addition to my headers hence the duplication of the Content-Type.

 

How can I force the Download tool to send “content-type = application/x-www-form-urlencoded" and consider only what is my Headers?

FrancoisP
7 - Meteor

Sorry, I meant:

 

How can I force the Download tool to not send “content-type = application/x-www-form-urlencoded" and consider only what is my Headers?

StephenF
8 - Asteroid

"To configure the iterative macro settings, you use the Interface Designer window."

 

Where is this Exactly?

 

desk.jpginterface designer -  Layout View"

 

 

shevshenko
7 - Meteor

@StephenF Probably you already found what you were looing for, but in case you haven't to find the Configuration Options for your Iterative Macro "To configure the iterative macro settings, you use the Interface Designer window."  - You just need to go to your left of that window where you see the 4 little icons aligned Vertically and select the last one, similar to a gear.

icon.JPG