Writing multiple sources to Database

lhk2000

Hi,

I have a Flow in alteryx that takes multiple files then creates a Database table, updates the table once, loads more data and updates(appends the table) the table. Rather than executing this flow in 2 steps. I would like to have a large "flow", where the next "batch" of data is appended AFTER the initial data is inserted into the new table.

How to complete this ?

Let me know if you need anything more.

Lars

Database Connection

Accepted answers

s_pichaipillai

you can wirte a expression in Filter tool and dont Grab them

All comments

s_pichaipillai

@lhk2000

are you looking for the option to INSERT only the new row which are not in Target Table from the source

1. you can refer the below link how you can achive this in standard workflow

http://blog.concentra.co.uk/2015/02/23/alteryx-perform-dynamic-incremental-data-extracts/

2. what database are you using, why don't you use In-DB tools and perform some join with TARGET table and comapre the data then INSERT the new rows

Let me know if this helps

lhk2000

Hi,

I am using IN - DB tool.

First i create the table then I union or Append the table with more data.

Working with fairly large amount of data ( more than 2 mill rows ).

s_pichaipillai

here you go

i use SQL Server table

its striaght method using Left join and grab the row that are now in target table, below is the target table structure

CREATE TABLE [dbo].[Sample](
	[ID] [varchar](254) NULL,
	[Order_Priority] [varchar](254) NULL,
	[Discount] [varchar](254) NULL,
	[Unit_Price] [varchar](254) NULL,
	[Shipping_Cost] [varchar](254) NULL,
	[Customer_ID] [varchar](254) NULL,
	[Ship_Mode] [varchar](254) NULL,
	[Product_Category] [varchar](254) NULL,
	[Product_Sub-Category] [varchar](254) NULL,
	[Product_Container] [varchar](254) NULL,
	[Product_Name] [varchar](254) NULL,
	[Product_Base_Margin] [varchar](254) NULL,
	[Order_Date] [varchar](254) NULL,
	[Ship_Date] [varchar](254) NULL,
	[Profit] [varchar](254) NULL,
	[Quantity_ordered_new] [varchar](254) NULL,
	[Sales] [varchar](254) NULL,
	[Order_ID] [varchar](254) NULL
) ON [PRIMARY]

hope this helps

InDB_lhk2000.yxzp

lhk2000

This looks very promising and is very close to what i want to achieve.

Thanks

lhk2000

Follow up question, in my data some of the columns are blank for certain rows and I would like to condition this that it excludes the entire row if ANY columns are blank. Any tip ?

Lars

Quick Links

Unanswered
Community Events
Groups

This months top contributors

AkimasaKajitani 387

mceleavey 383

mbarone 337

Hollingsworth 335

LanisC 335