Download a file only if not downloaded that day

Hi all,

As part of a workflow I am downloading a rather large file each time I run my workflow. As it takes a rather long time I am wondering if I could make the workflow in a way that it checks when the file has been last downloaded and if that is in the current day, it takes the previously downloaded file rather than downloading it again.

How could I achieve this in Alteryx?

Thanks in advance!

Nikki

Download

Accepted answers

danilang

Hi @Nikkib

Here's a workflow that shows what @jamielaird was referring to

1st run 1st run

The input tool reads the log db LastRun.yxdb(attached). The first time today, LastDate is less than today so the true branch of the filter is taken. The next tool, Block Until Down is there to ensure that your processing(output 1) completes successfully before the log file is updated(output 2). The log file update process just selects the LastDate field, sets it equal to today and writes it back to log db.

Subsequent runs

On subsequent runs on the same day, the bottom branch of the filter is taken and a message is display in the workflow messages window.

There's a small collapsed container at the top of the workflow that allows you to reset the date to yesterday. Just expand that container, collapse the main one and run the process.

Dan

If your files are saved locally or in a network drive, you could use the Directory Tool to see when the file was created/modified and build your workflow logic around that. Use the Dynamic Input tool if it was created today or download a fresh file if it was not.

All comments

jamielaird

Hi @Nikkib

You could do this by creating a log file that writes out the filename and current date/time each time the workflow runs. Then use this as a secondary input in your main workflow and add a filter tool that evaluates whether there is already a record in the log file for the current day. Connect the main workflow that includes the download tool to the 'True' output and an error message to the 'False' output.

Nikkib

Hi @jamielaird

Thanks! Could you maybe show how to make the log file. Pretty new to alteryx and not entirely sure what to do... 😄

danilang

Hi @Nikkib

Here's a workflow that shows what @jamielaird was referring to

1st run 1st run

Subsequent runs

On subsequent runs on the same day, the bottom branch of the filter is taken and a message is display in the workflow messages window.

There's a small collapsed container at the top of the workflow that allows you to reset the date to yesterday. Just expand that container, collapse the main one and run the process.

Dan

LastRun.yxdb

LastRun.yxmd

neilgallen

@Nikkib I might approach this in a different way, but there are some questions:

1) once the file has been downloaded, where is it stored? I'm assuming you have access to this location.

2) Where is the file being downloaded from?

Also, if you're just doing development, have you considered using the cache? It would prevent you from needing to download the file every time the workflow is run.

MichalM

@Nikkib

Quick Links

This months top contributors

atcodedog05 19598

Qiu 15880

binu_acs 15708

MarqueeCrew 13708

apathetichell 13703