Advent of Code is back! Unwrap daily challenges to sharpen your Alteryx skills and earn badges along the way! Learn more now.
Free Trial

Alteryx Designer Desktop Discussions

Find answers, ask questions, and share expertise about Alteryx Designer Desktop and Intelligence Suite.
SOLVED

Query to import subset of a YXDB file

Stephen_2
6 - Meteoroid

I have an input YXDB which contains around 6000 shape files and is quite large (~1.5gb).

I want to import this, filter out to a handful of relevant shapefiles and then do some processing. I can do this with a standard input and filter but then Alteryx needs to load the full 1.5gb input every time I run a query and takes ~ 5mins to load.

 

Is there any way to use a query on the input tool to only import the records of value? Similar to how you could use a SQL statement when importing from a database to pull a subset of all records.

 

Any help here is appreciated!

2 REPLIES 2
mattreynolds
9 - Comet

This could be a good use case for the Calgary database format which allows for indexing and further compression (see description below). Better compression and indexing means that even if you must read the entire file, the I/O requirements will be likely be lower (accelerating the queries you're running).

 

Happy to tweak a workflow if you can share.

 

Calgary.png

Stephen_2
6 - Meteoroid

Thanks Matt,

 

This has worked, took a while to convert to the Calgary DB but in future I'll just save that as the default output. Was able to do this query style operation with the Calgary Join. 

 

Appreciate your quick response 🙂 

Labels
Top Solution Authors