Happy 8th birthday to the Maveryx Community! Take a walk down memory lane in our birthday blog, and don't miss out on the awesome birthday present that all Maveryx Community members get to take advantage of!

Alteryx Designer Desktop Discussions

Find answers, ask questions, and share expertise about Alteryx Designer Desktop and Intelligence Suite.

Python Tool: Downloading Qualtrics Survey Data using Python API Calls

HeatherMHarris
11 - Bolide

I am thrilled to post this "one tool wonder" made possible by the new Python Tool in Alteryx Designer 2018.3.

 

Thank you to @IanCo, our Solutions Consultant, for steering me in this direction.

 

This post will show you how you can use the new Alteryx Designer Python Tool to implement the Qualtrics Python 3 API calls to download your survey data. (There is a template workflow attached below that includes all of the Qualtrics Python 3 code mentioned in this post for you to customize as follows.)

 

  • Open a new workflow and drag the Python Tool onto your canvas.

2018-09-21_15-45-15.png

 

 

 

  • Add the Qualtrics Python 3 code for the API calls just below the Python line of code that says "from ayx import Alteryx". The full code is at the bottom below so that you may cut-and-paste it into your Python Tool. (I found this Python 3 code on the Qualtrics web site: https://api.qualtrics.com/docs/response-exports. Make sure you select the tab for Python 3.)

 

 

2018-09-21_15-42-41.png 

 

 

 

  • Modify the next to last line of Qualtrics Python 3 code for the destination path for where you would like to land this data. It is landed in a directory as a CSV file.
    • You will need to put in your own file path on this line. (I added the 'r' before the path to allows to enter slashes without having to "escape" them as special characters 
      • zipfile.ZipFile(io.BytesIO(requestDownload.content)).extractall(r"YOUR_PATH_HERE")

 

IMPORTANT: Before you run your workflow, you need to make one change to a file in the Alteryx Designer 2018.3 Python installation.

 

  • Go to the following directory:
    • C:\Program Files\Alteryx\bin\Miniconda3\PythonTool_venv\Lib\site-packages\nbconvert\preprocessors
  • Edit the execute.py file
  • Change the following line from "30" to "-1" to disable the timeout -- The time to wait (in seconds) for output from executions.
    • timeout = Integer(-1, allow_none=True, ...

 

  • Run your workflow. The returned CSV file will be in the location that you entered above. There is nothing that comes into your workflow canvas. You can then use the Input Tool to read in the CSV file to a workflow.

 

Enjoy!

 

Heather Harris

Alaska Airlines

Alteryx ACE

@HeatherMHarris

 

 

----- Cut and Paste This Python Code Into the Python Tool -----

----- Enter your Qualtrics API Token, Survey ID and Data Center ID -----


# Python 3

 

import requests
import zipfile
import json
import io

 

# Setting user Parameters
apiToken = "<YOUR_TOKEN>"
surveyId = "<YOUR_SURVEY_ID>"
fileFormat = "csv"
dataCenter = '<YOUR_DATA_CENTER_ID>'

 

# Setting static parameters
requestCheckProgress = 0
progressStatus = "in progress"
baseUrl = "https://{0}.qualtrics.com/API/v3/responseexports/".format(dataCenter)
headers = {
    "content-type": "application/json",
    "x-api-token": apiToken,
    }

 

# Step 1: Creating Data Export
downloadRequestUrl = baseUrl
downloadRequestPayload = '{"format":"' + fileFormat + '","surveyId":"' + surveyId + '"}'
downloadRequestResponse = requests.request("POST", downloadRequestUrl, data=downloadRequestPayload, headers=headers)
progressId = downloadRequestResponse.json()["result"]["id"]
print(downloadRequestResponse.text)

 

# Step 2: Checking on Data Export Progress and waiting until export is ready
while requestCheckProgress < 100 and progressStatus is not "complete":
    requestCheckUrl = baseUrl + progressId
    requestCheckResponse = requests.request("GET", requestCheckUrl, headers=headers)
    requestCheckProgress = requestCheckResponse.json()["result"]["percentComplete"]
    print("Download is " + str(requestCheckProgress) + " complete")

 

# Step 3: Downloading file
requestDownloadUrl = baseUrl + progressId + '/file'
requestDownload = requests.request("GET", requestDownloadUrl, headers=headers, stream=True)

 

# Step 4: Unzipping the file
zipfile.ZipFile(io.BytesIO(requestDownload.content)).extractall("<YOUR_PATH>/MyQualtricsDownload")
print('Complete')

 

 

47 REPLIES 47
BobSnyder85
8 - Asteroid

Hey -- Just curious, does anyone know how to retain Chinese characters with this data call? I'm not familiar enough w/ Python to edit this code to force wide character support. right now i'm losing all my Chinese characters and they are being turned into goofy system characters.

 

It is possible this is happening upstream before it gets to qualtrics, and I will check that as well. Thanks in advance!

 

beckyfoulger
5 - Atom

I had the same issue. I'm not a python expert and this has all been frustrating, but I figured out how to keep special characters.  You just need to add a content parameter - see the green bold text below.

 

# Setting static parameters
requestCheckProgress = 0
progressStatus = "in progress"
baseUrl = "https://{0}.qualtrics.com/API/v3/responseexports/".format(dataCenter)
headers = {
"content-type": "application/json; charset=utf-8",
"x-api-token": apiToken,
}

 

Good luck! Now I am fighting a "Bad Zipfile" error that is only intermittent. 😞

BobSnyder85
8 - Asteroid

Thanks so much Becky!

 

awelz
5 - Atom

I have recently started getting SSLErrors with the workflow I created using this template.  All that was done was to update the data center and baseurl per the new documentation on qualtrics


See documentation here:

 

https://api.qualtrics.com/guides/docs/Guides/Common%20Tasks/getting-survey-responses-via-the-new-exp...

 

I have confirmed that the endpoint has a valid certificate. 

 

What am I doing wrong here?

 

SSLError: HTTPSConnectionPool(host='iad1.qualtrics.com', port=443): Max retries exceeded with url: /API/v3/surveys/SV_5cE7CxbxJKeWcLP/export-responses/ (Caused by SSLError(SSLError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:852)'),))

 

EW
11 - Bolide

Thanks so much to @HeatherMHarris and everyone else in this thread - this is incredibly helpful!  Another example of why Alteryx and its community are just the best!!  I just love this community.  

 

I'm also getting the SSL error mentioned by @awelz.  To avoid that, I added "verify=False" in the requests.  Now it runs and successfully downloads the survey responses (amazing!  I'm SO happy!) but it gives me a bunch of warnings like this:

 

Warning: Python (1): Unverified HTTPS request is being made to host 'xxxx.qualtrics.com'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings

 

I'm not very experienced with Python and just struggling with the Qualtrics API in general.  Does anyone know the best way of modifying the code to avoid the SSL error AND still have it verified / not cause the certificate verification error?  

Nethrap
5 - Atom

HI @HeatherMHarris,

 

I am trying to extract from multiple ID's, where as i am getting key error at one particular ID, when i run the same code for one same ID giving result, this is strange, can you please help me with that.

 

nomulap
5 - Atom

Hi Heather!

Thank you very much for the posting!

I do not see Survey Metadata and Embedded Data columns in the download except for ResponseId, Finished columns. Does your csv download contain all the Survey Medata and Embedded Data columns?

Regards,

Pavan

nomulap
5 - Atom

Hi Heather,

Thank you for the post!

How to download Survey Metadata and Embedded data also... attached code is not downloading this data.

 

TIA!
Pavan


Labels