community
cancel
Showing results for 
Search instead for 
Did you mean: 

Alteryx designer Discussions

Find answers, ask questions, and share expertise about Alteryx Designer.

Batch Process Files/Folders from Azure Data Lake

Meteor

We are storing thousands of files on Azure's Data Lake in nested folders.

 

With standard local folders the Alteryx file input tool combined with wildcards and the directory tool allows you to batch process files/folders recursively.

 

Does anyone know how to reproduce that functionality using nested files on Azure's Data Lake?

 

We have only been able to point to a specific file using the Azure Data Lake File Input tool.

 

Any help is appreciated!

Moderator
Moderator

Hello @WSData ,

 

You should be able to accomplish this using the Dynamic Lookup tool

 

This will allow you to set up multiple parameters on the kind of files it reads in to Alteryx.

 

Thanks!

TrevorS

Meteor

Trevor,

 

Thanks for the feedback.  We use the Dynamic input tool frequently with local files in conjunction with the directory tool to generate the recursive list of files.

 

However, when reading files from Azure Data Lake in the cloud, we did not see something that could generate a list of files in nested folders (like the directory tool for local files)

 

Have you been able to get the Dynamic Input tool to point directly to Azure Blob storage in the cloud and return a recursive list of files?

 

Thanks again.

 

 

Labels