community
cancel
Showing results for 
Search instead for 
Did you mean: 

Alteryx Designer Ideas

Share your Designer product ideas - we're listening!

Append to YXDB

It would be nice to be able to append to a YXDB instead of having to read in the whole thing, union the new records, and then re-create it. 
15 Comments
That would be a great feature as it would make much more efficient to add additional data to a yxdb dataset
Bolide
I second this - I need to append data to my data set each month and need a clean way to pull xml, parse and just append to previous month data.
Alteryx Certified Partner
Alteryx Certified Partner

This would be a game changer!

Founder
Founder
The YXDB file format is designed for some very specific needs around high speed reading and writing as well as having random I/O (for browsing) and a spatial index (for mapping/spatial join, etc...) This makes it not suited to more database like operations like append or query. When looking for a desktop file format that has database like functionality, SQLITE is the way to go. It is well supported in Alteryx, allows you to append/query etc. And then if you want indexes and stuff, you can build them with a separate tool and Alteryx will be able to use them.
Alteryx Certified Partner
Alteryx Certified Partner

Thanks for the tip ned, we will give SQLite a try today. It's really great.

 

I think a few of users were trying to read write and then append back to yxdb.

Alteryx Certified Partner
Alteryx Certified Partner

SQLLite works like a charm! Really encourage other users to try this.

Pulsar
Pulsar

Thank you Ned!  Been wondering about that!

Asteroid

Ned- Thanks for your response and I understand where you are coming from. One difficulty we face (and one that I bet is quite common) is that any setting up of additional software requires IT / admin rights for installation, maintenance, etc. I can't even download SQLite to try it out without IT's involvement. Hence, being able to append to the native Alteryx database would save us a fair amount of startup time. 

 

That said, we will look into SQLite as a way of appending, as this is a function we need. Our files are too large to continue appending as CSV, XLSX, but our organization isn't really big enough to maintain a true enterprise data warehouse.

Founder
Founder

You don't need to download SQLite.  It is built in to the Alteryx Input and Output tools.  No drivers or anything required.

Pulsar
Pulsar

Ned - correct me if I'm wrong, but I see about a 3x increase in file size if I go from YXDB to SQLLITE.  Does that seem right?  My input is simply a YXDB.  No tools in between.  Just a YXDB input and a SQLLITE output.  But the file size goes from about 50MB to 150MB.