This site uses different types of cookies, including analytics and functional cookies (its own and from other sites). To change your cookie settings or find out more, click here. If you continue browsing our website, you accept these cookies.
As part of a workflow I am downloading a rather large file each time I run my workflow. As it takes a rather long time I am wondering if I could make the workflow in a way that it checks when the file has been last downloaded and if that is in the current day, it takes the previously downloaded file rather than downloading it again.
You could do this by creating a log file that writes out the filename and current date/time each time the workflow runs. Then use this as a secondary input in your main workflow and add a filter tool that evaluates whether there is already a record in the log file for the current day. Connect the main workflow that includes the download tool to the 'True' output and an error message to the 'False' output.
Here's a workflow that shows what @jamielaird was referring to
The input tool reads the log db LastRun.yxdb(attached). The first time today, LastDate is less than today so the true branch of the filter is taken. The next tool, Block Until Down is there to ensure that your processing(output 1) completes successfully before the log file is updated(output 2). The log file update process just selects the LastDate field, sets it equal to today and writes it back to log db.
On subsequent runs on the same day, the bottom branch of the filter is taken and a message is display in the workflow messages window.
There's a small collapsed container at the top of the workflow that allows you to reset the date to yesterday. Just expand that container, collapse the main one and run the process.
If your files are saved locally or in a network drive, you could use the Directory Tool to see when the file was created/modified and build your workflow logic around that. Use the Dynamic Input tool if it was created today or download a fresh file if it was not.