INCREMENTAL LOAD

[Gopiroa]

## Incremental load

Iam trying to work on incremental load in Alteryx,iam having two tables with same data and column as we load by full load,suppose one table is dil_supplier_ds from oracle database and another table DIL_SUPPLIER_D from snowflake.
 NOW,IN DIL_SUPPLIER_DS WE UPDATE ONE ROW AND INSERT ONE ROW ,AND COLUMNS WE HAVE SUPPLIER KEY,COMPANY,LAST_UPDATE_DATE.

DIL_SUPPLIER_DS

SUPPLIER_KEY	LAST_UPDATE_DATE	COMPANY
DSP01	03/02/2002	ABC
DSP02	04/02/2002	ABCD
DSP03	08/05/2005	ABCDE
DSP04	08/08/2005	DELTA

SO USING FULL_INCREMENTAL LOAD I LOAD SAME TABLE INTO SNOWFLAKE,AS DIL_SUPPLIER_D.

DIL_SUPPLIER_D IN SNOWFLAKE

SUPPLIER_KEY	LAST_UPDATE_DATE	COMPANY
DSP01	03/02/2002	ABC
DSP02	04/02/2002	ABCD
DSP03	08/05/2005	ABCDE
DSP04	08/08/2005	DELTA

NOW ,I ADDED ONE ROW AND UPDATE DSP02 ROW.

THIS UPDATE TABLE IN DIL_SUPPLIER_DS IN ORACLE:

SUPPLIER_KEY	LAST_UPDATE_DATE	COMPANY
DSP01	03/02/2002	ABC
DSP02	08/08/2003	GIGA
DSP03	08/05/2005	ABCDE
DSP04	08/08/2005	DELTA
DSP05	08/08/2023	BELTA

SO I WANT A WORKFLOW,WHICH CAN INSERT A NEW ROW AND UPDATE A NEW ROW IN DIL_SUPPLIER_D IN SNOWFLAKE BY USING INCREMENTAL LOAD

###PLEASE PROVIDE A WORKFLOW FOR REFERENCE.

[ArnaldoSandoval]

Hi @Gopiroa 

You schema's tables should include at least these columns: "CREATION_DATE" and "LAST_UPDATE_DATE", you need the creation date column to identify new records in your incremental process, while the last update date assist you updating existing record on the target table.

hth

Arnaldo

[saraswathireddy]

We are loading data from an Oracle database into Snowflake using Alteryx. The full load process is working seamlessly. However, the issue arises when we attempt to implement the incremental load. Our source data contains a "LAST_UPDATE_DATE" column, and our goal is to compare this column in the target table to identify new records and updated records, then only update these specific records in the target table, achieving an incremental load.

We've gone through the Alteryx documentation, forums, and online resources to find a solution, but we have not been successful in resolving this matter. We believe that with your expertise and support, we can swiftly address this issue and ensure a successful incremental load process.

If you are not the appropriate person to address this matter directly, we kindly request that you guide us to the appropriate expert who can assist us in resolving this issue promptly. Given the timeline constraints due to our client demo, any assistance you can provide will be immensely appreciated.

If your appropriate person, please find more info  provided below

DIL_SUPPLIER_DS in oracle

SUPPLIER_KEY	LAST_UPDATE_DATE	COMPANY
DSP01	03/02/2002	ABC
DSP02	04/02/2002	ABCD
DSP03	08/05/2005	ABCDE
DSP04	08/08/2005	DELTA

 SO USING FULL_INCREMENTAL LOAD I LOAD THE SAME TABLE INTO SNOWFLAKE, AS DIL_SUPPLIER_D.

 DIL_SUPPLIER_D IN SNOWFLAKE

SUPPLIER_KEY	LAST_UPDATE_DATE	COMPANY
DSP01	03/02/2002	ABC
DSP02	04/02/2002	ABCD
DSP03	08/05/2005	ABCDE
DSP04	08/08/2005	DELTA

NOW, I ADDED ONE ROW AND UPDATE DSP05 ROW AND CHANGED THE COMPANY NAME TO GIGA(DSP02) IN THE SOURCE.(DIL_SUPPLIERS_DS)

SUPPLIER_KEY	LAST_UPDATE_DATE	COMPANY
DSP01	03/02/2002	ABC
DSP02	08/08/2023	GIGA
DSP03	08/05/2005	ABCDE
DSP04	08/08/2005	DELTA
DSP05	08/08/2023	BELTA

WE TO COMPARE : SELECT * FROM DIL_SUPPLIER_DS WHERE LAST_UPDATE_DATE>(SELECT MAX(LAST_UPDATE_DATE ) FROM DIL_SUPPLIER_D)

AFTER COMPARING THE ABOVE CONDITION, WE HAVE UPDATED THE DSP02 ROW AND DSP05

So, two Rows are to be  Updated in target based on condition.

 Thank you for your attention to this matter. We eagerly await your response, and we are committed to working collaboratively to overcome this challenge.

please help with workflow