ALTERYX INSPIRE | Join us this May for for a multi-day virtual analytics + data science experience like no other! Register Now

Alteryx Designer Discussions

Find answers, ask questions, and share expertise about Alteryx Designer.

Data Append and Delete using Alteryx yxdb

Vivekgiratkar
5 - Atom

I came across one requirement where there is around 23 M records at source((MS SQL) and every time i need to pull the data from SQL Server for alteryx workflow which is very time consuming .

I am looking for any way / better way to append data in .yxdb file in recent version of alteryx or any other better approach to append data.
Volume of data around 23 Millions and every day around 20 K record will be added.

MarqueeCrew
19 - Altair
19 - Altair
Rather than appending the data, have you considered creating a sub directory where you keep a yxdb file with names that include a date?

You can read the data with a wildcard and logically join as 1 file. As needed you can physically merge the data. Consider this a partitioned solution maybe?

Cheers,
Mark
Alteryx ACE & Top Community Contributor

Chaos reigns within. Repent, reflect and reboot. Order shall return.
Please Subscribe to my youTube channel.
Vivekgiratkar
5 - Atom

Thanks for your reply, but i do not want data to be maintain in separate yxdb file as from maintenance stand point we need to maintain data for last several year for analysis purpose, Any idea on appending data in calgary (.cycb ) or open source data SQLite, Any better approach which suits such requirement.

Labels