Alteryx designer Discussions

Find answers, ask questions, and share expertise about Alteryx Designer.

Python Tool: Downloading Qualtrics Survey Data using Python API Calls

Highlighted

I am thrilled to post this "one tool wonder" made possible by the new Python Tool in Alteryx Designer 2018.3.

 

Thank you to @IanCo, our Solutions Consultant, for steering me in this direction.

 

This post will show you how you can use the new Alteryx Designer Python Tool to implement the Qualtrics Python 3 API calls to download your survey data. (There is a template workflow attached below that includes all of the Qualtrics Python 3 code mentioned in this post for you to customize as follows.)

 

  • Open a new workflow and drag the Python Tool onto your canvas.

2018-09-21_15-45-15.png

 

 

 

  • Add the Qualtrics Python 3 code for the API calls just below the Python line of code that says "from ayx import Alteryx". The full code is at the bottom below so that you may cut-and-paste it into your Python Tool. (I found this Python 3 code on the Qualtrics web site: https://api.qualtrics.com/docs/response-exports. Make sure you select the tab for Python 3.)

 

 

2018-09-21_15-42-41.png 

 

 

 

  • Modify the next to last line of Qualtrics Python 3 code for the destination path for where you would like to land this data. It is landed in a directory as a CSV file.
    • You will need to put in your own file path on this line. (I added the 'r' before the path to allows to enter slashes without having to "escape" them as special characters 
      • zipfile.ZipFile(io.BytesIO(requestDownload.content)).extractall(r"YOUR_PATH_HERE")

 

IMPORTANT: Before you run your workflow, you need to make one change to a file in the Alteryx Designer 2018.3 Python installation.

 

  • Go to the following directory:
    • C:\Program Files\Alteryx\bin\Miniconda3\PythonTool_venv\Lib\site-packages\nbconvert\preprocessors
  • Edit the execute.py file
  • Change the following line from "30" to "-1" to disable the timeout -- The time to wait (in seconds) for output from executions.
    • timeout = Integer(-1, allow_none=True, ...

 

  • Run your workflow. The returned CSV file will be in the location that you entered above. There is nothing that comes into your workflow canvas. You can then use the Input Tool to read in the CSV file to a workflow.

 

Enjoy!

 

Heather Harris

Alaska Airlines

Alteryx ACE

@HeatherHarris

 

 

----- Cut and Paste This Python Code Into the Python Tool -----

----- Enter your Qualtrics API Token, Survey ID and Data Center ID -----


# Python 3

 

import requests
import zipfile
import json
import io

 

# Setting user Parameters
apiToken = "<YOUR_TOKEN>"
surveyId = "<YOUR_SURVEY_ID>"
fileFormat = "csv"
dataCenter = '<YOUR_DATA_CENTER_ID>'

 

# Setting static parameters
requestCheckProgress = 0
progressStatus = "in progress"
baseUrl = "https://{0}.qualtrics.com/API/v3/responseexports/".format(dataCenter)
headers = {
    "content-type": "application/json",
    "x-api-token": apiToken,
    }

 

# Step 1: Creating Data Export
downloadRequestUrl = baseUrl
downloadRequestPayload = '{"format":"' + fileFormat + '","surveyId":"' + surveyId + '"}'
downloadRequestResponse = requests.request("POST", downloadRequestUrl, data=downloadRequestPayload, headers=headers)
progressId = downloadRequestResponse.json()["result"]["id"]
print(downloadRequestResponse.text)

 

# Step 2: Checking on Data Export Progress and waiting until export is ready
while requestCheckProgress < 100 and progressStatus is not "complete":
    requestCheckUrl = baseUrl + progressId
    requestCheckResponse = requests.request("GET", requestCheckUrl, headers=headers)
    requestCheckProgress = requestCheckResponse.json()["result"]["percentComplete"]
    print("Download is " + str(requestCheckProgress) + " complete")

 

# Step 3: Downloading file
requestDownloadUrl = baseUrl + progressId + '/file'
requestDownload = requests.request("GET", requestDownloadUrl, headers=headers, stream=True)

 

# Step 4: Unzipping the file
zipfile.ZipFile(io.BytesIO(requestDownload.content)).extractall("<YOUR_PATH>/MyQualtricsDownload")
print('Complete')

 

 

Highlighted
Alteryx Certified Partner
Alteryx Certified Partner

Hi @HeatherHarris this is great; did you ever try implimenting this before the python tool? Assume it can be made possible using the download tool?

 

Ben

Highlighted

Hi @BenMoss,

 

I was actually working on an implementation using the Download Tool last week. I was working through some technical details with our Solutions Consultant, @IanCo, when he suggested the Python Tool.

 

There are three Qualtrics API calls required to complete the data request, monitoring and download process. It appeared you would need to use three separate download tools -- and implement and manage the data transfer, monitoring and orchestration between them. The Python Tool was a faster, simpler implementation.

 

Heather

@HeatherHarris

Highlighted
Alteryx
Alteryx

Looks great Heather! Very helpful!

Highlighted

Hi Heather, thank you. It works like a charm. Awesome!

Highlighted
Meteoroid

Thanks for the helpful guide - Works like a charm!

Highlighted
Atom

 

Unfortunately I still get a timeout error in Alteryx after changing the execute.py file.  Any other suggestions?  Screenshots below:

 

 

Execute.py updateExecute.py updateErrorError

 

 

 

 

Highlighted
Asteroid

is anyone using this? I am getting an error

 

     26 downloadRequestPayload = '{"format":"' + fileFormat + '","surveyId":"' + surveyId + '"}'
     27 downloadRequestResponse = requests.request("POST", downloadRequestUrl, data=downloadRequestPayload, headers=headers)
---> 28 progressId = downloadRequestResponse.json()["result"]["id"]
     29 print(downloadRequestResponse.text)
     30 

KeyError: 'result'
Highlighted
Alteryx Certified Partner

Hi Jack,

 

Tool runs clean for me now.  As the first hurdle to overcome, to simply use the Python tool at all I first needed to change the Temp Directory that was configured on my machine.  I found this by going to Options > User Settings > Edit User Settings > Defaults.  The Python tool would not run correctly until I had set this temp directory to "C:\ProgramData\Alteryx\Engine"

 

Now, under the basic conditions the tool runs clean and works correctly.  I do not get that error message that you mention.

 

However, I do receive that exact same error message when attempt to modify the API call in Step #1.  I want to modify the downloadRequestPayload to send the parameter "useLabels": True.  This parameter instructs to display the text labels for each question that are shown to the user (rather than the internal numbers that are defined in the survey).

 

So, because I get that error message when I do something incorrect with the syntax of my API call, I will suggest that you can try again by downloading the generic template tool and being very careful to only modify the python code in correct locations.

 

Follow up question:

 

Is anyone able to please assist me with the correct syntax to use in the downloadRequestPayload section in order to specify "useLabels": True ?

 

Please confirm.

Thanks!

 

Highlighted
Alteryx Certified Partner

Hi everyone,

 

I am attaching here the template python code, now modified for using the survey response download API with "useLabels": True.  As instructed above, you must still make your edits to this template code and provide your own surveyID, apiToken, dataCenter, and zipfile extract location.

 

The UseLabels parameter is set to either True or False, and will dictate whether or not the survey responses are downloaded with Answer Text Labels or with Recode Values. There is not a way to export both the Answer Labels and the Recode Values with one single call.  So if you want both, then the work around would be to do two exports, and join the datasets on your end.

 

Also, separately:

 

In my own implementation of the tool, it was easy to add a # Step 5 to the end of the script.  This step reads the survey response contents out of the downloaded CSV, and sends them as output from the python tool for further downstream processing in the same Alteryx workflow.

 

# Step 5: Read CSV into Data Frame
df = pd.read_csv("<YOUR_PATH>\<YOUR_CSV_FILE>.csv", sep=",")
Alteryx.write(df,1)

 

Hope this helps.

Thanks!

Labels