Read files from directory where there are close to 700+ files which has unique name for each file suffix with the Date, should consider only the latest available file ( going back upto 3 days from today) from the field where the date field is suffixed
sample
BARX_20250228.csv
BARX_20250303.csv
BARX_20250304.csv
BARX_20250305.csv
GHJ_20250303.csv
GHJ_20250304.csv
Using right function - got the date from the file name to new field
using Right function separated the date to a new field
@disub I am not sure if you asked a question or not, but one way of getting the latest 3 files, is first to extract the dates from the file name, convert them into actual date, then sort decent, using a sample tool extract the first 3 values
@disub ,
Assuming you already get the list of csv file names (with Directory Tool or whatever),
you can parse the date part in the file name and filter with the date function.
Formula to parse date part
date_in_suffix =
DateTimeParse(
REGEX_Replace([FileName], ".*_(\d{8})\.csv", "$1"),
"%Y%m%d"
)
Filter to get dates within 3 days from today
DateTimeDiff(
DateTimeToday(),
[date_in_suffix],"day"
) <= 3
I hope this helps
used formula tool to bring the Date suffixed with the file name and converted to Date format & added Filter function " dateTimeDiff(DateTimeToday(),[DateTime_Out],'day')<=3 . Now on the "True" - it considers the latest 3 files from the list and on " False" the other backed dated file.
The next step is to consider the 1 latest file each from the "TRUE" function. How to make it consider 1 latest each file?
You can sort the date and then use a Sample tool to select the first of x group's no. 1 row.
sort tool picks file latest randomly but not considering all the required files from the directory