Community Spring Cleaning week is here! Join your fellow Maveryx in digging through your old posts and marking comments on them as solved. Learn more here!

Alteryx Designer Desktop Knowledge Base

Definitive answers from Designer Desktop experts.

APIs in Alteryx: cURL and Download Tool

MattD
Alteryx Alumni (Retired)
Created

API connections give access to many web-based applications, database systems, or programs by exposing objects or actions to a developer in an abstracted format that can easily be integrated into another program. In other words, an API functions as a point of integration where you can programmatically access or manipulate information for a program (or workflow) you’re building. In the context of Designer use, APIs are most often used to access data stores, utilize web-hosted services while blending, or to build connectors.

 

Most APIs are in RESTful architecture, the software architectural style of the World Wide Web, and typically communicate over HTTP with the same HTTP actions, or “verbs”, that web browsers use to retrieve web pages and transfer data: GET, POST, PUT, DELETE, among others. As such, any tool or software that gives you the ability to use the HTTP request-response protocol will let you communicate with these RESTful APIs. The three I most commonly use to establish my connections are Postman (a great testing app that resembles the Alteryx Download Tool configuration), cURL (a command line tool and one of the predominant HTTP/FTP transfer libraries), and everybody’s favorite software: Alteryx. Between these three tools, and maybe a web traffic monitor like Fiddler (this will let you see the HTTP requests being sent/received over your network), you should be able to establish, maintain, and automate connectivity to just about any REST API of your choosing. This article briefly exposes how to do so in cURL and Alteryx so that you can (1) easily pivot between the two for more robust troubleshooting/implementation and (2) more readily implement API connections into your Alteryx workflows, even when they are documented programmatically.

 

The first step in establishing an API connection is to locate the developer documentation. The quality and detail of documentation for an API is often the limiting reagent of establishing a connection, so be sure to use an API that has the best functionality, then documentation. The documentation will then help walk you through the API’s authentication before introducing the different requests you can make. Since cURL is frequently used amongst API developers, you’ll notice many of the example requests you are introduced to will be in cURL syntax. Being able to decipher those cURL requests will help you easily recreate it in a program with an interface, like Postman or Alteryx. Consider the GET and POST requests below (all included in the attached v10.6 workflow for reference – please note that the workflow will not run due to redacted credentials to the API):
 

 

1.png

2.png


The documentation gives us a URL to communicate with via an HTTP action (two in this case, GET and POST), specific to this request of listing users on search criteria: http://community.lithium.com/community-name/restapi/vc/search/users. It also notes a parameter, or argument, to accompany the request – those keywords are most often associated with the request payload. Along with a payload argument for authentication (redacted), we’ll use these elements to generate our requests. For the cURL requests, feel free to use the executable installed along with your Alteryx Designer, located in C:\Program Files\Alteryx\bin\RuntimeData\Analytic_Apps\ by default (you can get your own download here).

 

GET

Request in cURL (command line):

 

3.png

(the –i option is used to include the response headers to resemble the Alteryx format below)

 

Request in Alteryx Download Tool:

 

4.png

 

POST

Request in cURL (command line):

 

5.png

 

Request in Alteryx Download Tool:

 

6.png

 

Note: A series of cURL commands can also be implemented into a .bat file to be executed within a workflow via the Run Command Tool(master it here).


Translating the Requests
Above you can see side by side examples of cURL syntax and how the API request would look in the Designer. While we can’t go over every type of request you may need, we can equip you with a metaphorical pocket dictionary that will help you translate between the two:

 

Lookup Table.jpg

 

 

  • The URL in your cURL request will simply need to make its way to the URL Field in the Basic Tab of the Download Tool.
  • Timeout time (--connect-timeout in cURL) can be changed in the Connection Tab.

The schematics below identify some of the elements from our lookup table, but in syntax: 
 

 

Request 1.jpg

Request 2.jpg

 

Additionally, Postman, the tool mentioned earlier, can build requests much like in Alteryx and has a “Generate Code” feature that will convert the request to usable cURL syntax that can be helpful in translating.

 

While the above helps elucidate some of the similarities between cURL and Alteryx HTTP requests, there are also some notable specificities to using each. For example, cURL will give you far more control over the more granular configuration options for each request; things like what version of HTTP to use, passing cookies, using proxies, among others. Alteryx, on the other hand, gives you the flexibility to build an entirely different request from each row of data entering the Download Tool, making it near effortless to generate a large number of requests customized to your data or to automate API interactions that would otherwise require daunting programming. In addition, Alteryx makes it far easier to parse an API response into usable data such that it can be blended with your other datasets – all inside the same workflow that made the requests. Depending on the types of requests you’ll be making to your API, you’ll have to look at the different formats above and determine the optimized approach. For example, what if you need very specific request configuration but see yourself making a large number of requests? You could use Alteryx to automate writing all the syntax of your cURL commands to a .bat file, and then run that .bat file in a Run Command Tool from Alteryx. Now that you know both, choose wisely!



Additional Resources

 
Attachments
Comments
TashaA
Alteryx Alumni (Retired)

Great article on cURL requests @MattD!

ryan_pip
6 - Meteoroid

Hi Matt, Have you had any luck getting the download tool to work with kerberos.  I know for curl you just need to add a --negotiate -u but can't seem to find the equivalent for the Download tool

joeschafer
6 - Meteoroid

I've had a lot of success with APIs that are GET based and only require payload arguments.


I've had zero success with APIs that are GET based and have header requirements, unless those headers are able to fit into the normal URL structure, like https://api.example.com/header1/header2/callthisapi?

 

there are examples out there, documented for cURL, that don't seem to fit into that sort of paradigm.  I'll see something like:

 

curl "https://www.google.com/search?"
  -H "Authorization: Basic $BASE64_ENCODED(API_USERNAME:API_PASSWORD)"


Which, aside from the Base64 encoding part (there's a tool for that!), don't make sense to me in terms of how to take a cURL example and turn it into a header suitable for the Download tool.

 

 

esar
8 - Asteroid

This is a priceless article. Thank you so much @MattD

kal-tech
6 - Meteoroid

Hi Matt - I am trying to connect "Salesforce Marketing Cloud(SMC)" using Alteryx.  The idea is to download some "Campagn" data from SMC. Any suggestions how to approach ?  I have Alteryx "download" tool in designer.

MaryMargret
5 - Atom

 Hi 

I am trying to access VSTS API URL (Azure) from Alteryx using download tool. Normal configuration with email id password couldnt establish the connection. Then while searching came accross the use of Personal Access token. Could you please share the sample code if you have done something similar to this.

 

Regards,

Margret

davidhenington
10 - Fireball

Wish we could delete comments :). 

 

I figured it out. 

npandya
7 - Meteor

What about adding _K/--insecure in download tool? Or if we can accomplish it with Run command tool and not the download tool then how

GreggD
8 - Asteroid
What about working with Rest calls to a secure HTTPS URL? I see nothing about how to handle SSL Authentication. Anyone have any ideas?
ryan_pip
6 - Meteoroid
I simply use the python tool now for all my api needs. Handles, kerberos, insecure and https and you can even use asyncio to handle large volumes of calls. If you do end up using asyncio it can only run in production mode as otherwise the jupyter's own loops get in the way.
npandya
7 - Meteor
huynv96
9 - Comet

How i do if curl like this: curl –i -u account_id:api_key "https:anwebsite.api"?

field url: https:anwebsite.api

How i include "account_id:api_key"?

fed_
7 - Meteor

@huynv96 did you find out how to do so? 
I have a very similar issue

huynv96
9 - Comet

@fcalanca my endpoint API have another get request method, something like add parameter client=account_id:api_key. You can read API document or require support. I think almost API support it.

sidgrowup
5 - Atom

hi,

 

have anyone had a chance to track USPS delivery, using his API, with download and json tool.

SideOfRanch
8 - Asteroid

Does anyone know how you would convert this cURL string into the download tool?

 

curl --header "Private-Token: <your_access_token>" "https://gitlab.example.com/api/v4/projects"

 

I'm trying to convert the above curl authentication into the download tool and can't figure out how to place it in the header tab of the settings for the download tool.

Ritesh127
6 - Meteoroid

Thank you for this! The above article is very helpful in understanding how curl commands can be translated into the download tool

 

It states the Get/put, - d (for data) and - H (for headers) equivalents in the download tool, but is there a -F equivalent available in the download tool - for the purpose of uploading a file via api?

 

Would appreciate any thoughts on this!

Thank you

jrobiso2
8 - Asteroid

Error: Download (1): Error transferring data: SSL peer certificate or SSH remote key was not OK

 

When trying to do API via the Download tool.  Any way around this?

 

The Self-Signed cert we are using is from the previous incarnation of the server, so the hostname is wrong, and yes, I have added the cert to the Trusted Root Certification Authorities in the OS.

 

Unlike Curl, where I can use -k, there doesn't appear to be any way to do the same in the Download Tool.

 

My intent is this: Initial connection to API uses a POST https request with username/pass in raw-data JSON format to get issued a Bearer Token. Then in the workflow, subsequent API requests will use that Bearer Token from the very first API call as the required auth token. But this gets me nowhere if I can't get the darned https protocol to ignore any ssl validation!

severhart
8 - Asteroid

Has anyone used this for a connection to an API provided from Visier? When I initially put the connectors into Excel I have to enter a userid/password - how can that be included with this process? Would LOVE to solve for this!

jrobiso2
8 - Asteroid

I found that you can use an Interface and the Download tool to do just about anything you need. I made an example, but I can't seem to find a button to allow attaching the file here.

taylor_made
Alteryx
Alteryx

Extremely helpful, thank you so much. 

Sean_Walker
Alteryx
Alteryx

Hey @ryan_pip  did you have an example of how you used the Python tool to connect to a Kerberos auth endpoint? 

AlteryxUserFL
11 - Bolide

I’m trying use the Alteryx download tool to extract data from an API. I tried following this post to translate the CURL to the fields need for the download tool, but I keep getting a HTTP/1.1 400 Bad Request. Below is a CURL that I’m using and my attempt to translate per the post for the download tool. Does anyone have any pointers? The * symbols were replaced with real data in Alteryx. 

 

 curl -X POST "https://api.openpeoplesearch.com/api/v1/Consumer/PhoneSearch" -H  "accept: text/plain" -H  "Authorization: Bearer ****************************************" -H  "Content-Type: application/json" -d "{\"phoneNumber\":\"**********\"}"

 

 

 

URL https://api.openpeoplesearch.com/api/v1/Consumer/PhoneSearch

Header1 accept: text/plain

Header2 Authorization: Bearer: ****************************************"

Header3 accept: Content-Type: application/json"

Payload: Content-Type: {\"phoneNumber\":\"**********\"}

jrobiso2
8 - Asteroid

Did you try curl -k -X . . . . .  the -k might do the trick.