Input data from a new directory with consistent folder structure

[bkoons]

Can someone help me find a solution to this seemingly common problem?

Every month my team has to run a process contained in a consistently defined directory. For example, the process run in January 2017 has files contained in the following sample directory structure:

Parent folder:

c:\2017_Jan\

subfolders:

input\

output\

analysis\

The files and sub-folders under the directory are named the same each month. The files that feed my workflow are in the same sub-folders each month. The only thing that changes is the main directory folder. For example:

c:\2017_Jan\input\INPUT_A.csv

c:\2017_Feb\input\INPUT_A.csv

etc.

Side note: I think I want the parent directory to be a text input because it will be quicker to change the small portion of the file path than to point to the folder.

Thanks in advance!!

[JordanB]

Hi @bkoons

This article uses the directory tool to read in the latest file using the Creation date field. You could do something similar by using this field depending on when the saves get saved in that folder. Or you can parse the date out of the file path, then sort and filter those files. 

An example of parsing the date is attached.

Best,

Jordan Barker

Solutions Consultant

[bkoons]

Thanks for the reply, unfortunately you're working in a more recent version of Alteryx and I can't open your solution. I should have stated initially I'm working in 10.6. Our IT group controls which version we have installed so I don't have the option to upgrade yet either.

Also, the solution will need to be flexible enough where we could go back and run prior data, so we wont always be running the most recent files.

[JordanB]

Hi @bkoons

See attached for 10.6.

Feel free to remove the sample tool and replace with a filter so you can see both latest and previous files.

Best,

Jordan Barker

Solutions Consultant

[Coxta45 · Accepted solution]

@bkoons,

I think the dynamic input tool will be your friend here.  Try something like this:

Use a Text Input to declare some variables.  You'll notice you can easily update the "parent_directory" before each run.  This field could also very easily be updated via an action tool if you decide to create an application.

Then follow the Text Input with a Formula tool and create a new field called "full_path" that builds the full filepath...

Lastly, use the Dynamic Input tool to read the appropriate file.  Use the "Change Entire File Path" option, replacing with your new field like so:

Notice you'll need to select a template file as a placeholder - which in your case could be a current copy of INPUT_A.csv

Cheers,

Taylor

[tom_montpool]

Using a Text Input to declare some variables is one way...and a good one.

I thought I should point out that you can set Workflow Constants in the Workflow Configuration and you can access the constants in other tools (e.g. Filter, Formula).

[bkoons]

This is an excellent tip that I will definitely use. Thanks!

[bkoons]

Thanks Taylor, I think this is what I need to do. I apologize, the Alteryx notification emails were caught up in my company spam filter so I didn't realize others had responded. 

[bkoons]

@Coxta45

Taylor, in your solution, the Input Data Source template must be pointed to a directory. I'm not sure I will be allowed to point, for example, from the February directory back to the January directory. Or to go from a testing environment to production will require me to go in and manually change all of those templates. Is there a way around this? One of my thoughts is can I import a template into the workflow? 

[Coxta45 · Accepted solution]

@bkoons,

The Input Data Source template is exactly that, a template.  Meaning it can be any copy of the template saved in any location that Alteryx can read.  It could simply be a file called "My data template" saved on your desktop.  As long as the file structure matches that of the dynamic ones that will be read, the tool should work.  The tool is going to read from the location that it gets from the variables, etc.

My recommendation, given the example above, would be to put a template @ "C:\INPUT_A_Template.csv" - and so on as needed - that are simply copies of a previous month's file.  Or you could just use an actual previous month's file in the template input, so long as the file will always be there...