community
cancel
Showing results for 
Search instead for 
Did you mean: 

Alteryx designer Discussions

Find answers, ask questions, and share expertise about Alteryx Designer.

How to get data from Azure Blob storage to Alteryx?

Atom

Hi!

 

I'm trying to get data (JSON-files) from a Blob-storage in MS Azure. I have the Account Name and Shared Key for the account as well as know the container name. How can I get that data into Alteryx?

 

BR,

 

Arttu

Alteryx
Alteryx

Hi @matinlaa

 

There is currently no native connectivity from Alteryx to Azure Blob storage.  This is the first request we have gotten for Azure Blob storage connectivity.  I will recommend posting on our Product ideas forum so that our Products Team can have visibility on this request and track it in terms of priority over other requests.  The more stars an idea has the more likely it is to be considered.

 

https://community.alteryx.com/t5/Alteryx-Product-Ideas/idb-p/product-ideas

 

That said, I am not very familiar with Azure Blob storage connectivity from external products.  However, typically our Download tool will retrieve data from a specified URL to be used in downstream processing or to be saved to a file which also supports blob type fields.   However, I will recommend reading the following information regarding our downloads tool in case this can be of use depending on your connection options from Azure Blob Storage.  

 

Downloads Tool Help:

https://help.alteryx.com/11.0/index.htm#Download.htm  

 

Downloads Tool Community Article:

https://community.alteryx.com/t5/Alteryx-Knowledge-Base/Tool-Mastery-Download/ta-p/29583

 

Christian Rincon
Manager, Customer Support
Alteryx, Inc.
Nebula
Nebula

Hey @matinlaa - how did you fare with this one - did you mange to get to a solution with @ChristianR 's suggestions?

Thought I'd throw in an update.  I have been able to use the download tool to grab files out of Azure blob storage, from both publicly accessible containers as well as private containers using an SAS key.  In your storage account page, under Shared Access Signatures, it will generate the SAS token to include on the URL for you.  Then just set your URL field in the download tool to: https://mystorageaccount.blob.core.windows.net/mycontainername/myfile.csv and either include the SAS token on the URL, or include it on the "Headers" tab in the download tool.

 

The resulting file comes back just like the results of any other download tool (in its original format but stored in a single field) so you'll need to parse it back into a data object.

 

The other option is to use the Run Tool to execute azcopy (with the storage accounts key) or curl (with an SAS token) from the commandline to actually pull down the file in its original format, then pull it into your stream using the Run tool.

 

For bigger (GB) files or with slow connections, the Run tool seems more reliable, but your mileage may vary!

Alteryx
Alteryx
Atom

Hello,

 

I tried following the guidance in the above post and could not get the blob upload to work. Azure Blob storage is unable to accept a PUT request due to the following:

 

HTTP/1.1 411 Content-Length HTTP header is missing.

 

As a result, the blob cannot be uploaded. Please advise on resolution. Thanks in advance.

 


@lee2017 wrote:

Hello,

 

I tried following the guidance in the above post and could not get the blob upload to work. Azure Blob storage is unable to accept a PUT request due to the following:

 

HTTP/1.1 411 Content-Length HTTP header is missing.

 

As a result, the blob cannot be uploaded. Please advise on resolution. Thanks in advance.

 



I have the same issue

So my original post only dealt with downloading from azure storage blobs since you can do that through a URL pretty easily.  Uploading files to blob storage is another story.

 

The easiest way to do so is using a utility called AzCopy.exe, which is free from microsoft.  It is a command line utility that you can incroporate into Alteryx using the Run Command macro.  Here is some AzCopy documentation.

 

I've attached an App that lets you post files to an azure blob by specifying all of the required parameters (the azcopy.exe filepath, blob account url, bucket/container name, and destination key).  I had to go back and sanitize some stuff out of it, so I haven't verified that this version works exactly, but it should work for you.  You can then take the workflow and see if you can't reverse engineer it into a macro you can use in stream.

 

Let me know if this gets you where you're trying to go!

Meteoroid

Hi David

 

I am brand new to alteryx but have an immediate need to be able to do this.  could you possibly share some screen shots?  i have been trying for the last 2 hours and keep coming up against problems.  Any help would be very much appreciated.

 

thanks

 

Highlighted
Meteoroid

Since there is no Alteryx connector available for Azure storage, the best way I have found to get the data out of Azure is to use Azure Data Factory and output it to a local file server.  

 

Not ideal, but it will work in a pinch.

Labels