Updating Multiple Input Files Automatically

Hi All,

I have a workflow which has multiple input files (around 8) and those data get updated every day. Each of them are separate data sets and have different file names.

Is there a way Dynamic Input can be used to get the Input files updated whenever a new data of the same input file is stored in the system?
Any help would be highly appreciated.

Thanks

Datasets

Connectors

Transformation

Dynamic Processing

Accepted answers

All comments

caltang

Sorry, I may not be entirely sure if I understand you... but if you're putting it in the same location, you can use * asterisk to call all files in + a Batch Macro (if needed - data structure being different from each file).

Could you share more details?

db89

@caltang No worries but thanks for your response! Let me elaborate, for example there are 2 inputs:

1.Revenue File which has columns like Client,Invoice,Fee,Margin

2.Expense File which has columns like Invoice, Cost, Payment details
Now the those 2 files would have different dates at the end of the file names as those are extracted daily (for ex. Revenue File_7/1/23; Revenue File_7/2/23....)

But, as input files I have to change them because I need to select the most updated data set but do not want to combine them by using " * " in the input directory.

Hence my query is, is there a way that Dynamic input can help?

Qiu

@db89
I suppose you meant that when there is an input file is updated, alteryx workflow will be triggered?

If that is the case, a new feature called "watch dog" was introduced at the Alteryx Inspire 2023.
it is an event-driven scheduling featureb but we are not sure when it will be released.

db89

@Qiu basically i want to use the updated data file after i store the most updated file in a folder, for ex. Revenue File_7/1/23 is the older file and today I downloaded Revenue File_7/2/23 so, basically i have to go into the Input tool and select the latest file.

So my question is whether Dynamic input tool can help with fetching the updated input file or not.

caltang

Besides Dynamic Input, I would go with the Batch Macro option regardless whether they are the same data frame / format to input the sheets per file. After the Batch Macro calls all the files in, what you can do is add a RecordID to the sheets imported - then you can use an interface via Analytic App to select the latest files OR if you name your files accordingly, you can Parse out the date, and then add a filter tool to be dynamic to call the dates you want or # of files you want (per latest upload).

Not sure if this helps, but it's just an idea.

Qiu

@db89
Sorry about the misunderstanding.
How about using a Directory tool given the target directory contains only the files you want.

jdminton

@db89 Yes! I use this in several workflows. Use the directory tool as mentioned above to pull in the files. There are a couple different ways you can determine which files are new and only include these.

The first way (if you are only trying to bring in a current day's file) is to filter based on the date using the date in the directory tool. You could also bring in the latest date if that works better for your application. Depending on your data, you may be able to use a date in the data to determine the dates needed

If you add the filename to the data when you use the dynamic input, you can use that filename to identify new files and only pull the new files. Use a simple join tool on the filename will do this. If you don't already have filename in your data, you can set a filter for going forward and add the filename to the new data only.

You can give these methods a try or provide us with some sample data you are using to provide more specific help.

Quick Links

This months top contributors

atcodedog05 19598

Qiu 15867

binu_acs 15708

MarqueeCrew 13708

apathetichell 13703