Incremental Load

Question

Hi everyone

I am fairly new to Alteryx and wanted some help regarding the below workflow. The data is a "select * from table" from an Azure table which is then loaded to Snowflake and also a CSV is generated. It's a full load each time (runs daily) and I have to change it to incremental load because the workflow takes ages. I have been asked to use Dynamic Input tool. Can anyone please guide me on this?

Screen Shot 2021-09-01 at 1.33.13 pm.png

Sebastiaandb · Answer

Hi

@priyaraghuveer

Here you go, i just took a specific part of the flow as there's a lot of confidential info in it :-). Hope it helps!

Vandaag is "Today" in Dutch :-).

priyaraghuveer · Answer

Hello,

Thank you, do you mind sharing a screenshot of the workflow please?

Sebastiaandb · Answer

HI @priyaraghuveer

I'm not familiar with Azure. However, i do know the solution for this problem in a Oracle table with sql.

What i think you can do (depends on which fields you have available in your database table):

1- First setup a standard sql connection to your azure table in the dynamic input tool. Make sure you're having a field that marks all fields from a certain date/time. (f.e. AZURE_DATETIME_FROM > 31-08-2021.

2- Now you select "replace a specific string" by pressing the add button in the dynamic input tool.

3- In order to make this work, you have to have a field from the incoming stream that you can utilize to replace the string. In this case you want to change the 31-08-2021 by a new date (take in mind formatting by using f.e. DateTimeFormat). You can make a date field in the formula tool by using DateTimeFormat(DateTimeToday(), f, l ), in f you can give up the format you'll need for your sql and l is optional so not needed for sql).

Now it should work.