Can I tell Alteryx to look at the filenames, and just run the workflows from a directory?

[nborzil1]

Hello - 

CONTEXT: I am working on a pretty sizable workflow to manually stage data for an ETL. it is a series of 13 small WFs to combine 13 reports into one. Each report contains Protocol, Site, Subject, Cohort - so this portion of the task is very easy.

However I am getting hung up on automating the task. Each file name every week is the same, but it's different for every trial. Sometimes the filenames overlap.

ISSUE/QUESTION: The current process is download all reports, rename them with the study's protocol ID, place anywhere, open Alteryx, manually drag anD drop each file to it's corresponding workflow, set it up, then run. Once complete, decomission input files in preparation for next week's upload.

See what I mean? it's too onerous. Instead of using a static input file, can I tell Alteryx to look at the filenames, and just run the workflows from a directory, that contain a string, (run x workflow FROM file containing "xx.csv")?

ALSO: The end file, is staged for upload via ETL. It is a cumulative report, containing past studies. Every week the process is repeated, so this has to match that. Old data can't be erased. I'm thinking i should separate the legacy data, join it with the current workflows, and then have the end, output file have the same name as the staging file, so it can immediately be staged for ingestion.

I hope I explained the process enough, let me know if you have questions. Since this is for my work, I need to be careful to anonymize the screenshots and workflows. All data is blinded however.

[mceleavey]

Hi @nborzil1 ,

What you need is the Directory tool combined with a Dynamic Input tool, or, as I have used, my  macro that imports multiple flat files in a single tool:

I've attached the exported workflow and attached it.

I hope this helps,

M.

[nborzil1]

Thank you for getting back so soon! I think this is much closer to what I was aiming for.

A concern I have is that the input files are all slightly different - some of them are a specific report, and others are custom, from a template. So the columns will be in different orders, and may have slightly different names - the primary reason why all these workflows are separate and in containers.

Would your solution still work? Or is it designed to repeat the same WF for all files?

[mceleavey · Accepted solution]

Hi @nborzil1 ,

It will work, but you will need to do a separate one for each TYPE of file. That is, say you have the first three are the same format, then they can all be loaded in together, the next three are the same, so they would be done together etc. You may simply need to copy and paste my section and rename the file in the directory tool to only pick up the ones you want.

M.

[nborzil1]

I see - so worst case scenario I'd need to add it to each container, which isn't so bad. I will fiddle with this when I get time. Thank you for your suggestions!

[mceleavey]