Python Tool: Downloading Qualtrics Survey Data using Python API Calls

[HeatherMHarris]

I am thrilled to post this "one tool wonder" made possible by the new Python Tool in Alteryx Designer 2018.3.

Thank you to @IanCo, our Solutions Consultant, for steering me in this direction.

This post will show you how you can use the new Alteryx Designer Python Tool to implement the Qualtrics Python 3 API calls to download your survey data. (There is a template workflow attached below that includes all of the Qualtrics Python 3 code mentioned in this post for you to customize as follows.)

• Open a new workflow and drag the Python Tool onto your canvas.

• Add the Qualtrics Python 3 code for the API calls just below the Python line of code that says "from ayx import Alteryx". The full code is at the bottom below so that you may cut-and-paste it into your Python Tool. (I found this Python 3 code on the Qualtrics web site: https://api.qualtrics.com/docs/response-exports. Make sure you select the tab for Python 3.)

 

• Modify the following three lines of the Qualtrics Python 3 code for your particular Qualtrics token, survey and data center values.
  • apiToken = "YOUR_API_TOKEN_HERE"
  • surveyId = "YOUR_SURVEY_ID_HERE"
  • datacenter = 'YOUR_DATA_CENTER_ID_HERE"
• Qualtrics provides some help for finding these values on the following web sites:
  • https://api.qualtrics.com/docs/root-url
  • https://api.qualtrics.com/docs/finding-qualtrics-ids

• Modify the next to last line of Qualtrics Python 3 code for the destination path for where you would like to land this data. It is landed in a directory as a CSV file.
• You will need to put in your own file path on this line. (I added the 'r' before the path to allows to enter slashes without having to "escape" them as special characters 
• zipfile.ZipFile(io.BytesIO(requestDownload.content)).extractall(r"YOUR_PATH_HERE")

IMPORTANT: Before you run your workflow, you need to make one change to a file in the Alteryx Designer 2018.3 Python installation.

• Go to the following directory:
  • C:\Program Files\Alteryx\bin\Miniconda3\PythonTool_venv\Lib\site-packages\nbconvert\preprocessors
• Edit the execute.py file
• Change the following line from "30" to "-1" to disable the timeout -- The time to wait (in seconds) for output from executions.
  • timeout = Integer(-1, allow_none=True, ...

• Run your workflow. The returned CSV file will be in the location that you entered above. There is nothing that comes into your workflow canvas. You can then use the Input Tool to read in the CSV file to a workflow.

Enjoy!

Heather Harris

Alaska Airlines

Alteryx ACE

@HeatherMHarris

----- Cut and Paste This Python Code Into the Python Tool -----

----- Enter your Qualtrics API Token, Survey ID and Data Center ID -----

# Python 3

import requests
import zipfile
import json
import io

# Setting user Parameters
apiToken = "<YOUR_TOKEN>"
surveyId = "<YOUR_SURVEY_ID>"
fileFormat = "csv"
dataCenter = '<YOUR_DATA_CENTER_ID>'

# Setting static parameters
requestCheckProgress = 0
progressStatus = "in progress"
baseUrl = "https://{0}.qualtrics.com/API/v3/responseexports/".format(dataCenter)
headers = {
    "content-type": "application/json",
    "x-api-token": apiToken,
    }

# Step 1: Creating Data Export
downloadRequestUrl = baseUrl
downloadRequestPayload = '{"format":"' + fileFormat + '","surveyId":"' + surveyId + '"}'
downloadRequestResponse = requests.request("POST", downloadRequestUrl, data=downloadRequestPayload, headers=headers)
progressId = downloadRequestResponse.json()["result"]["id"]
print(downloadRequestResponse.text)

# Step 2: Checking on Data Export Progress and waiting until export is ready
while requestCheckProgress < 100 and progressStatus is not "complete":
    requestCheckUrl = baseUrl + progressId
    requestCheckResponse = requests.request("GET", requestCheckUrl, headers=headers)
    requestCheckProgress = requestCheckResponse.json()["result"]["percentComplete"]
    print("Download is " + str(requestCheckProgress) + " complete")

# Step 3: Downloading file
requestDownloadUrl = baseUrl + progressId + '/file'
requestDownload = requests.request("GET", requestDownloadUrl, headers=headers, stream=True)

# Step 4: Unzipping the file
zipfile.ZipFile(io.BytesIO(requestDownload.content)).extractall("<YOUR_PATH>/MyQualtricsDownload")
print('Complete')

[HeatherMHarris]

@action-analytics:

Thank you for this excellent addition to the solution!

Now, if someone would make it into a macro (I don't have the time or a Qualtrics account anymore), it would be awesome.

Anyone out there accept the challenge? ;-)

@HeatherMHarris 

[fredy_arias]

useLabels True

# Step 1: Creating Data Export
downloadRequestUrl = baseUrl
useLabels = True
dictionaryPayload = {'format': fileFormat, 'surveyId': surveyId, 'useLabels': useLabels}
downloadRequestPayload = json.dumps(dictionaryPayload)
downloadRequestResponse = requests.request("POST", downloadRequestUrl, data=downloadRequestPayload, headers=headers)
progressId = downloadRequestResponse.json()["result"]["id"]
print(downloadRequestResponse.text)

[kevisull]

I added the Python tool.

I added the code.

I added my pieces: 

apiToken = "xxxxxxxx"
surveyId = "xxxxxx"
fileFormat = "csv"
dataCenter = 'xxxxxxx.co1.qualtrics.com'

File Path:  (r"\\xxx\xxx\546\Alteryx Data Files\MyQualtricsDownload")

I changed the execute.py file to show -1

When I run the workflow it runs for 5 seconds no errors but also no results....  What am I missing?  this is my first time using the python tool.

Thanks

@IanCo 

@HeatherMHarris 

[gene_denny]

I'm wondering if anyone has updated this code with the new Python 3 code provided, and now required, by Qualtrics.  Aside from being able to spell it, I know nothing about Python other than I could copy, paste, and edit the code provided earlier to use with our own surveys.  Could not be more thankful for those involved in the initial code!  Could use some help if anyone is able to take the new code set and apply needed settings as I am getting nothing but very generic errors.  Here is the link to the new code provided by Qualtrics if it's helpful.

 https://api.qualtrics.com/docs/survey-export-new-guide

[HeatherMHarris]

Hello, @gene_denny !

I'm so glad that the original solution was helpful!

Unfortunately, I no longer work with Qualtrics data and am unable to create and test a new solution.

If I haven't missed anything, the only two lines in the new code you need to edit are:

surveyId = "SV_6hsqQZccKfnIKGN"
dataCenter = 'co1'

You would put in your own surveyID and datacenter values.

Maybe someone who has an active Qualtrics account can chime in as well.

Cheers!

Heather

[gene_denny]

Thanks for the response!  

I've tried updating the APIKEY, the datacenter (which is the same) and the SurveyID, but I get errors still that mean nothing to me, so I'm still stuck.  Thanks for taking a look, Heather.  Sure wish you still worked with this!

[gene_denny]

I've decided to post my full code here to see if anyone is able to ascertain where the error might be in my syntax.  I have replaced the APIKey and surveyID with xxxxxxxx, however any punctuation such as single or double quotes are exactly as they appear in my code.  My previous post shows the error being rendered.  All of this is happening via the use of the Python tool contained within the Developer tool set.  I'm guessing this is something very simple but again, I don't code python at all.  Based on the rendered error, it would appear that I need to define the environment variable but I don't know how to do this.

Thanks ahead of time to anyone that can help!  This is big for our organization!

[HeatherMHarris]

@gene_denny  I would consider filing a support ticket on this one, as the error makes me think it's possible that the Jupyter notebook interface is not interacting well ... maybe not closing the code execution session???

[gene_denny]

Heather, have I told you how much I appreciate you?  CUZ I DO!

I assume you're talking about opening a case with Alteryx, not with Qualtrics?  Qualtrics has made it pretty clear they will provide the base code but no support after that point.

[HeatherMHarris]

Yes, this one may merit some Alteryx support. 🙂

Will you please share the solution here once you figure it out?

Thanks!

Heather