Incremental Load

[priyaraghuveer]

Hi everyone

I am fairly new to Alteryx and wanted some help regarding the below workflow. The data is a "select * from table" from an Azure table which is then loaded to Snowflake and also a CSV is generated. It's a full load each time (runs daily) and I have to change it to incremental load because the workflow takes ages. I have been asked to use Dynamic Input tool. Can anyone please guide me on this?

[Sebastiaandb]

HI @priyaraghuveer 

I'm not familiar with Azure. However, i do know the solution for this problem in a Oracle table with sql. 

What i think you can do (depends on which fields you have available in your database table):

1- First setup a standard sql connection to your azure table in the dynamic input tool. Make sure you're having a field that marks all fields from a certain date/time. (f.e. AZURE_DATETIME_FROM > 31-08-2021.  

2- Now you select "replace a specific string" by pressing the add button in the dynamic input tool.

3- In order to make this work, you have to have a field from the incoming stream that you can utilize to replace the string. In this case you want to change the 31-08-2021 by a new date (take in mind formatting by using f.e. DateTimeFormat). You can make a date field in the formula tool by using DateTimeFormat(DateTimeToday(), f, l ), in f you can give up the format you'll need for your sql and l is optional so not needed for sql). 

Now it should work. 

[priyaraghuveer]

Hello, 

Thank you, do you mind sharing a screenshot of the workflow please?

[Sebastiaandb]

Hi

@priyaraghuveer 

Here you go, i just took a specific part of the flow as there's a lot of confidential info in it :-). Hope it helps!

Vandaag is "Today" in Dutch :-).