We are aware of an issue with the Search bar. Please use Advanced search for the time being while we troubleshoot. Thanks for your patience as we work on improving the community!

Alteryx Designer Discussions

Find answers, ask questions, and share expertise about Alteryx Designer.

Data Append and Delete using Alteryx yxdb

5 - Atom

I came across one requirement where there is around 23 M records at source((MS SQL) and every time i need to pull the data from SQL Server for alteryx workflow which is very time consuming .

I am looking for any way / better way to append data in .yxdb file in recent version of alteryx or any other better approach to append data.
Volume of data around 23 Millions and every day around 20 K record will be added.

19 - Altair
19 - Altair
Rather than appending the data, have you considered creating a sub directory where you keep a yxdb file with names that include a date?

You can read the data with a wildcard and logically join as 1 file. As needed you can physically merge the data. Consider this a partitioned solution maybe?

Alteryx ACE & Top Community Contributor

Chaos reigns within. Repent, reflect and reboot. Order shall return.
Please Subscribe to my youTube channel.
5 - Atom

Thanks for your reply, but i do not want data to be maintain in separate yxdb file as from maintenance stand point we need to maintain data for last several year for analysis purpose, Any idea on appending data in calgary (.cycb ) or open source data SQLite, Any better approach which suits such requirement.