Writing multiple sources to Database

[lhk2000]

Hi,

I have a Flow in alteryx that takes multiple  files then creates a Database table, updates the table once, loads more data and updates(appends the table) the table. Rather than executing this flow in 2 steps. I would like to have a large "flow", where the next "batch" of data is appended  AFTER the initial data is inserted into the new table.

How to complete this ?

Let me know if you need anything more.

Lars

[s_pichaipillai]

@lhk2000

are you looking for the option to INSERT only the new row which are not in Target Table from the source

1. you can refer the below link how you can achive this in standard workflow

http://blog.concentra.co.uk/2015/02/23/alteryx-perform-dynamic-incremental-data-extracts/

2. what database are you using, why don't you use In-DB tools and perform some join with TARGET table and comapre the data then INSERT the new rows 

Let me know if this helps

[lhk2000]

Hi,

I am  using IN - DB tool.

First i create the table then I union or Append the table with more data.

Working with fairly large amount of data ( more than 2 mill rows ).

[s_pichaipillai]

here you go 

i use SQL Server table

its striaght method using Left join and grab the row that are now in target table, below is the target table structure

CREATE TABLE [dbo].[Sample](
	[ID] [varchar](254) NULL,
	[Order_Priority] [varchar](254) NULL,
	[Discount] [varchar](254) NULL,
	[Unit_Price] [varchar](254) NULL,
	[Shipping_Cost] [varchar](254) NULL,
	[Customer_ID] [varchar](254) NULL,
	[Ship_Mode] [varchar](254) NULL,
	[Product_Category] [varchar](254) NULL,
	[Product_Sub-Category] [varchar](254) NULL,
	[Product_Container] [varchar](254) NULL,
	[Product_Name] [varchar](254) NULL,
	[Product_Base_Margin] [varchar](254) NULL,
	[Order_Date] [varchar](254) NULL,
	[Ship_Date] [varchar](254) NULL,
	[Profit] [varchar](254) NULL,
	[Quantity_ordered_new] [varchar](254) NULL,
	[Sales] [varchar](254) NULL,
	[Order_ID] [varchar](254) NULL
) ON [PRIMARY]

hope this helps

[lhk2000]

This looks very promising and is very close to what i want to achieve.

Thanks

[lhk2000]

Follow up question, in my data some of the columns are blank for certain rows and I would like to condition this that it excludes the entire row if ANY columns are blank. Any tip ?

Lars

[s_pichaipillai · Accepted solution]

you can wirte a expression in Filter tool and dont Grab them