Listening for new files in a directory to initiate workflow

[whyman]

Curious if anyone has figured out how to monitor the contents of a directory (could be ftp, sftp folders) to kickoff a workflow as new files are added to the directory?

Any help is appreciated!

[chris_love · Accepted solution]

The way I've done this in the past is to keep a list of file already processed in a table or file, then run the module every 5 minutes (or more often), and pull down the full list from the site / directory and compare. Kick off a process on the new files then record them as processed in the table..

[whyman]

Chris,

Thanks for the reply.  

I have not tested the scheduling tools...is that what you are utilizing to run the workflow every 5 (or x) minutes?

[chris_love]

Yes exactly, we have lots of processes like this. e.g. we currently run a process every minute that logs where planes are around the world, we build up a minute by minute picture of where they are by continually adding to a database. One example, but shows how easy it is to implement.

[Mervin1975]

How would you start this. What node would I use?

[wymanb]

I believe Chris was suggesting using a directory tool to gather file info at a point in time, then persist that info in a file/database etc. Then on a regular cadence (I.e. 5 min) run the directory tool and compare the results to what you persisted. The difference are new files that need to be processed. Rinse and repeat.

[Mervin1975]

I was looking to create a task or job in Alteryx to check the directory for files placed there and process them if they are found.