Import Multiple Folders with One Directory Tool?

[ddiesel]

I'm starting a new project today and I'm working on bringing in files from multiple folders.

Example Directory Structure:

If my objective is to bring in everything from folders containing "ABC", can it be done without using multiple directory tools? There are hundreds of sub folders... not just three.

Can this be accomplished in one step?

[Garrett · Accepted solution]

How about a more open ended Directory tool, followed by a Filter tool?

Ignore the error; I don't have a C:\Data\ folder.

[ddiesel]

@Garrett

So this works but it's so slow. 36 minutes to process due to the number of sub folders and contents.

Is there a way to use a Directory tool to find the folders I need and then bring in full path of the contents of those folders?

[JoshuaM]

@Bolide

I believe you can optimize @Garrett's solution to only include folders specific to "ABC" (see below).

I had a similar situation and only wanted a File Specification of "Jan".

Using the below approach I retrieved the full path for files within the folder "01 Jan" and sub folder labeled "Jan 01".

There was another sub folder labeled "Feb 02" which was neglected.

This should hopefully help optimize your workflow as I believe the original approach would have to bring in every field to filter out those containing "ABC"

[pbrink]

Is there a way once you have the files you need to then push them out to a new directory or folder? Thanks

[JoshuaM]

@pbrink

Yes, the way I handled this was to edit the "Directory" output field from the Directory tool.

Add a summarize after the Directory tool and group by the Directory field.

Then just dynamically replace the fields within the current file path.

(Below I simply removed the current folder "Submissions" and essentially saved the data as a sheet ("Data Source") in an existing excel template.) 

 

The final step is to just have an Output tool which references the new Directory field. 

When configuring the tool just select "Take File/Table Name From Field" & "Change Entire File Path".

Hope this helps,

 Josh