Append Tool - Confirm My Understanding

Question

Hi. I have a .yxdb file that is about 6 million rows of data that encompasses 7 months of daily activity. I have a workflow that pulls in the prior day data of about 5,500 rows and joins it back to the historical yxdb. The process takes a while because it's writing to the Tableau hyper file and to xydb.

I see the Append Tool, but it appears that it does not just add one day of data to the xydb (like we do in creating/writing tables in DBase). There is nothing in Alteryx that appends the data, correct? That is, everytime I want to add a day's worth of data I have to bring in the historical data and union it to the one day of data and re-write the entire xydb. Correct? I am just double checking as I am struggling to get this workflow to run on the Gallery as it just takes forever and times out, but runs just fine in the Designer.

alexnajm · Answer

Yes it does @JPSeagull! I am not sure what you mean exactly by "basic structure" but you'd have your historical table already there and you'd just have to pass the new records into it to append.

Hyper Append Snap.PNG

JPSeagull · Answer

@alexnajm - hyper files has an append option? Really? How would that look in a basic structure?

JPSeagull · Answer

Thank you, @alexnajm . xlsx files won't help here because of the row count being over 1 million.