Community Spring Cleaning week is here! Join your fellow Maveryx in digging through your old posts and marking comments on them as solved. Learn more here!
The Product Idea boards have gotten an update to better integrate them within our Product team's idea cycle! However this update does have a few unique behaviors, if you have any questions about them check out our FAQ.

Alteryx Designer Desktop Ideas

Share your Designer Desktop product ideas - we're listening!
Submitting an Idea?

Be sure to review our Idea Submission Guidelines for more information!

Submission Guidelines

Snowflake Bulk Loader

It would be really helpful to have a bulk load 'output' tool to Snowflake.  This would be functionality similar to what is available with the Redshift bulk loader.

Currently it takes a reaaally long time to insert via ODBC or would require you to write a custom solution to get this to work.

 

This article explains the general steps but some of the manual steps outlined would have to be automated to arrive at a solution that is entirely encapsulated within a workflow.

http://insightsthroughdata.com/how-to-load-data-in-bulk-to-snowflake-with-alteryx/

23 Comments
jroot
8 - Asteroid

 @BradW Are there any plans to be able to use Snowflake's built-in staging area? I don't think it's uncommon to have folks like me that have Snowflake without access to another S3 bucket. We're trying to use Snowflake so we don't have to mess with AWS directly. Love the direction Alteryx is going in though!

ARich
Alteryx Alumni (Retired)

Hi @jroot,

 

Thanks for the feedback! There aren't currently plans to use Snowflake's built in staging area, but that's exactly the type of feedback we're hoping for so that we can add that to Snowflake bulk if there is enough demand for it!

 

Best,

Alex

Kirk
5 - Atom

I absolutely agree that there should be an option to use the internal Snowflake stage.  Otherwise users are forced to have a separate Amazon S3 account (which Snowflake by itself does not require) and pay for those additional storage and miscellaneous costs.  Thanks!

fpinchon
8 - Asteroid

I agree on the built-in staging as well, table stages are very simple to code, since each table has its own in Snowflake, and to use it, just have to add @% in front of the table name...

https://docs.snowflake.net/manuals/user-guide/data-load-local-file-system-create-stage.html

 

danners
7 - Meteor

+1 on utilizing the built-in snowflake stage.  Having to provision a separate S3 bucket is just an additional step before an end-user can get started and isn't always possible due to internal controls or fiscal constraints.

Please! :) 

tkreykes
6 - Meteoroid

Another vote for built-in snowflake stage

robshiffer
7 - Meteor

One more vote for built-in snowflake staging. Is there an expected timeline for when the non-beta version will be released?

jkruger
6 - Meteoroid

Another vote for built-in snowflake stage

ARich
Alteryx Alumni (Retired)
Status changed to: Implemented

HI All,

 

Snowflake Bulk on AWS has been released. Please start a new Idea thread for further bulk functionality requests. I do see that a lot of people are asking for bulk via built-in staging, which we'll look into, but would also be very helpful to track as a separate request!

 

Thanks for all of the feedback!

 

Best,

Alex

fpinchon
8 - Asteroid

Hi Alex,

 

This is definitely progress. But S3 is so finicky for analysts like me, a version of the connector that would use Snowflake internal staging area is really a MUST HAVE.

 

I am experiencing config issues with the new S3 connector, and I am not alone:

https://community.alteryx.com/t5/Alteryx-Designer-Discussions/Problems-loading-data-using-Snowflake-...

 

Is there a way to provide better help than this:

https://help.alteryx.com/2018.3/DataSources/Snowflake.htm?Highlight=bulk

 

or at least some visibility into the script that the connector is generating?

 

Another ask would be to make the Data connection editable! There are so many fields to input, and each time you press Save, it cannot be edited anymore! A real nightmare when trying to debug that connection by trying different combinations of extractions...