Reading Files H

[Jaspal80]

I have a folder which has lots of files and every day new files gets created in the folder.

I am creating alteryx workflow to read most recent excel files which got saved today.

The file name are different for each file but all file name end with YYYYMMDD which is Today - 2( T-2,  so on 26th Oct the files will be saved with 20221024 name in it)

file name 

xxxx.xxx xxx - xx.xxx.20221024

How can I read all the files with ends with most recent date - in the above case the latest date would be 20221024 and next day the file will be 20221025 and so on. 

Also, since there are lots of files and saved in another server it takes a while to read the files from this folder. Any way we can improve the performance to read the files?

Thank you for your help.

[Carlithian]

You can use a directory tool to identify what is in the folder, then a filter with your date logic to ensure only the files you care about continue downstream and feed them into a dynamic input tool to bring the data into the workflow 

[Felipe_Ribeir0]

Hi @Jaspal80 

Take a look at the attached example workflow, it is doing exactly that for .txt files. Just adapt the directory input tool and the dynamic input tool to your case and the more recent YYYYMMDD file will be the source.

[Sebastiaandb]

Hi @Jaspal80 , 

Short answer would be to built a batch macro that reads files based on a control parameter that is fed in a workflow by a directory tool and having a filter tool (if you only want files from datetimenow()-2 then you could state Datetimeparse(right([FIELD],8),"%Y%m%d") = Datetimeadd(datetimenow(),-2,"days"). Alternatively, instead of a filter tool, you can also make a new field from the directory tool with something like right([FIELD], 8), followed by a summarize tool with max on that new field so that you have the last date available. Then you would join this back with the rows from the directory tool and the "new" FIELD (with the right forumula), the result will be only the rows from the files of the latest date. You can then feed the batch macro with the paths of these files. 

Hopefully this is a bit clear... I have a example batch macro attached.

Greetings,

Seb

[Felipe_Ribeir0]

Hi @Jaspal80 

It worked?