Append to YXDB

[mbarone]

It would be nice to be able to append to a YXDB instead of having to read in the whole thing, union the new records, and then re-create it. 

[Ned]

The YXDB file is compressed which makes it smaller.  That compression is part of what makes it so you can't append to one.  The actual compression will vary based on what data you write to it.  SQLite is appendable and queryable which inherently would be very difficult or impossible to support compression.

[mbarone]

Got it . . . thanks for all the info!

[MarshallG]

Very cool. Thanks, Ned!

[anthony]

Nice - glad I checked back in on this thread.

I was using a local mysql instance the last few days so I could insert 200k parsed xml files...1 at a time. It was way to slow doing this with read-union-write yxdb method, so I started using mysql and hated the fact I needed to add another technology to the stack.

This will work great to build my data and then export as a yxdb for faster down stream processing for other processes.