Download a file only if not downloaded that day

[Nikkib]

Hi all, 

As part of a workflow I am downloading a rather large file each time I run my workflow. As it takes a rather long time I am wondering if I could make the workflow in a way that it checks when the file has been last downloaded and if that is in the current day, it takes the previously downloaded file rather than downloading it again. 

How could I achieve this in Alteryx?

Thanks in advance! 

Nikki 

[jamielaird]

Hi @Nikkib 

You could do this by creating a log file that writes out the filename and current date/time each time the workflow runs. Then use this as a secondary input in your main workflow and add a filter tool that evaluates whether there is already a record in the log file for the current day. Connect the main workflow that includes the download tool to the 'True' output and an error message to the 'False' output.

[Nikkib]

Hi @jamielaird 

Thanks! Could you maybe show how to make the log file. Pretty new to alteryx and not entirely sure what to do... 😄 

[danilang · Accepted solution]

Hi @Nikkib 

Here's a workflow that shows what @jamielaird was referring to 

1st run 

The input tool reads the log db LastRun.yxdb(attached).  The first time today, LastDate is less than today so the true branch of the filter is taken.  The next tool, Block Until Down is there to ensure that your processing(output 1) completes successfully before the log file is updated(output 2).  The log file update process just selects the LastDate field, sets it equal to today and writes it back to log db.

Subsequent runs

On subsequent runs on the same day, the bottom branch of the filter is taken and a message is display in the workflow messages window.

There's a small collapsed container at the top of the workflow that allows you to reset the date to yesterday.  Just expand that container, collapse the main one and run the process.

Dan

[neilgallen]

@Nikkib I might approach this in a different way, but there are some questions:

1) once the file has been downloaded, where is it stored? I'm assuming you have access to this location.

2) Where is the file being downloaded from?

Also, if you're just doing development, have you considered using the cache? It would prevent you from needing to download the file every time the workflow is run.

[MichalM · Accepted solution]

@Nikkib 

If your files are saved locally or in a network drive, you could use the Directory Tool to see when the file was created/modified and build your workflow logic around that. Use the Dynamic Input tool if it was created today or download a fresh file if it was not.