Connecting to Redshift via views or direct query?

Question

Hi all,

I'm importing data from a Redshift database and am wondering what the fastest way is. Do I create views in the database and use those to import data into Alteryx or do I use a direct query?

Does anybody have experience with that and willing to share their view on the matter?

Thanks!

Silmaril300 · Answer

Hi Brandon,

Thank you for your informing reply.

A bit more about what I'm trying to achieve. I'm taking customer data from Redshift, perform a Churn analysis and write it back into a Azure Data Lake.

The amount of data we're dealling with is quite large (several tables with ~100 000 000 rows). At the moment we're reading the data from Redshift and saving it as a alteryx database but I'm still wondering if there is a more clever way.

Also, if we decide to use a view, some of the data wrangling steps could be done in the database itself (I know you can use the direct query to do so as well but again, would that make a difference in efficiency?).

At the moment, we're not looking at streaming the data but run the workflow once per week (month).

Thanks again for your insight and help!

BrandonB · Answer

You also have the option to use In-DB processing with the tools in the In-DB category: https://help.alteryx.com/current/designer/amazon-redshift

Additionally, Alteryx has bulk loading capability for Redshift for fast writing back up into the database.

The "fastest way" will depend on what you are attempting to accomplish with the workflow. For example, it may be beneficial to take local sources and data stream them into Redshift as temp tables and then join against the tables using In-DB tools. Or you may need to stream the data out at some point as well depending on what your output objective is. Hopefully this helps!