Upload csv to Bigquery with download tool

Im trying to upload some csv files into google bigquery with the download tool (I couldn't get the output tool to upload with the simba drivers).

I have successfully got the authentication sorted, but when i try to either upload a multipart POST option i get a '413 query too large error' while when I try to set up a resumable upload (that would probably be more useful) the connection times out when getting the initial resumable session.

Any ideas for what I'm doing wrong?

Connectors

Accepted answers

paul_houghton

So I managed to work out how to do this and the key issue I had was realising I needed to use the blob family of tools. (I went with the blob input and take the file from an temporary csv file).

I have published the tool on the gallery and would love more feedback on any errors that people might find so I can make it more robust.

https://gallery.alteryx.com/#!app/Big%2BQuery%2BTable%2BUpload/576ba6a9aa690a12b4e1279c

Thanks

All comments

JCR

@GarthM & team. Any thoughts?

NeilR

Could you share your workflow (removing the data and auth credentials)?

paul_houghton

So I managed to work out how to do this and the key issue I had was realising I needed to use the blob family of tools. (I went with the blob input and take the file from an temporary csv file).

I have published the tool on the gallery and would love more feedback on any errors that people might find so I can make it more robust.

https://gallery.alteryx.com/#!app/Big%2BQuery%2BTable%2BUpload/576ba6a9aa690a12b4e1279c

Thanks

shreyanshrathod

Hi @paul_houghton @NeilR

Warm Greetings!!

Can you guide me into analysing best practices to follow while uploading data to BigQuery from Alteryx Designer ?

I am employing Google BigQuery Output tool for the same, but have little clue about best adjusting the configurations of it.

The problem is because of huge data size. My team member ran a job to upload data to BQ with a default batch size of 500 and after running for 13 hours, the job failed!!

There is very less documentation for Alteryx connectivity with BQ!

Any insights from your end will be really helpful.

Thanks in advance!!

Best Regards,

Shreyansh Rathod

Quick Links

This months top contributors

atcodedog05 19598

Qiu 15878

binu_acs 15708

MarqueeCrew 13708

apathetichell 13703