Community Spring Cleaning week is here! Join your fellow Maveryx in digging through your old posts and marking comments on them as solved. Learn more here!
The Product Idea boards have gotten an update to better integrate them within our Product team's idea cycle! However this update does have a few unique behaviors, if you have any questions about them check out our FAQ.

Alteryx Designer Desktop Ideas

Share your Designer Desktop product ideas - we're listening!
Submitting an Idea?

Be sure to review our Idea Submission Guidelines for more information!

Submission Guidelines

Append to YXDB

It would be nice to be able to append to a YXDB instead of having to read in the whole thing, union the new records, and then re-create it. 
15 Comments
adrian_loong
7 - Meteor
That would be a great feature as it would make much more efficient to add additional data to a yxdb dataset
anthony
11 - Bolide
I second this - I need to append data to my data set each month and need a clean way to pull xml, parse and just append to previous month data.
adrianloong
11 - Bolide

This would be a game changer!

Ned
Alteryx Alumni (Retired)
The YXDB file format is designed for some very specific needs around high speed reading and writing as well as having random I/O (for browsing) and a spatial index (for mapping/spatial join, etc...) This makes it not suited to more database like operations like append or query. When looking for a desktop file format that has database like functionality, SQLITE is the way to go. It is well supported in Alteryx, allows you to append/query etc. And then if you want indexes and stuff, you can build them with a separate tool and Alteryx will be able to use them.
adrianloong
11 - Bolide

Thanks for the tip ned, we will give SQLite a try today. It's really great.

 

I think a few of users were trying to read write and then append back to yxdb.

adrianloong
11 - Bolide

SQLLite works like a charm! Really encourage other users to try this.

mbarone
16 - Nebula
16 - Nebula

Thank you Ned!  Been wondering about that!

MarshallG
8 - Asteroid

Ned- Thanks for your response and I understand where you are coming from. One difficulty we face (and one that I bet is quite common) is that any setting up of additional software requires IT / admin rights for installation, maintenance, etc. I can't even download SQLite to try it out without IT's involvement. Hence, being able to append to the native Alteryx database would save us a fair amount of startup time. 

 

That said, we will look into SQLite as a way of appending, as this is a function we need. Our files are too large to continue appending as CSV, XLSX, but our organization isn't really big enough to maintain a true enterprise data warehouse.

Ned
Alteryx Alumni (Retired)

You don't need to download SQLite.  It is built in to the Alteryx Input and Output tools.  No drivers or anything required.

mbarone
16 - Nebula
16 - Nebula

Ned - correct me if I'm wrong, but I see about a 3x increase in file size if I go from YXDB to SQLLITE.  Does that seem right?  My input is simply a YXDB.  No tools in between.  Just a YXDB input and a SQLLITE output.  But the file size goes from about 50MB to 150MB.