How to filter a YXDB input during input

[alex_porreca]

I am using an input with hundreds of millions of rows, but only need a limited range of that yxdb file. Is there a way to filter it similar to a where clause in an SQL statement?

[afv2688]

Hello @alex_porreca ,

YXDB is designed for fast simple interchange of data and has no indexes. Therefore I don't think you are able to perform the task you are looking for.

Regards

[CharlieS · Accepted solution]

---

@alex_porreca wrote:

I am using an input with hundreds of millions of rows, but only need a limited range of that yxdb file. Is there a way to filter it similar to a where clause in an SQL statement?

---

You'll need to write an Alteryx Calgary file (.cydb) instead of a .yxdb. Calgary files support field indices so each field can be quickly searched and filtered. A Calgary Join tool can then use these indices to quickly return results that join/match your criteria in your workflow. 

You can read more on this documentation page, or just ask here!

https://help.alteryx.com/2018.3/calgaryindex.htm

@afv2688 wrote:

YXDB is designed for fast simple interchange of data and has no indexes.

---

Spatial fields are indexed by default in .yxdb files. There's a checkbox to disable this on the Output tool. 

[afv2688]

You are right @CharlieS, thanks for correcting me and preventing me misinform someone. Will get you a beer at Inspire!😄🙂